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ABSTRACT 


The  purpose  of  this  thesis  is  to  estimate  the  potential  perfonnance  improvement  in 
sustaining  engineering  (SE)  when  an  Open  Architecture  (OA)  approach  to  system 
development  is  used.  Its  basis  is  that  in  Integrated  Warfare  Systems  (IWS)  acquisition, 
eighty  percent  of  total  life-cycle  costs  occur  during  the  operation  and  support  phase.  This 
statistic  demonstrates  the  necessity  of  measuring  how  the  OA  approach  will  affect 
software  upgrade  and  maintenance  process  for  the  AEGIS  IWS  Life  Cycle.  Using  the  OA 
approach,  advances  in  distance  support  and  monitoring,  and  maintenance  free  operating 
periods  are  possible,  and  this  is  significant  in  supporting  the  need  to  reduce  costs  and 
manpower  while  improving  performance.  To  estimate  the  potential  (Return  on 
Investment)  ROI  that  an  OA  approach  might  enable  for  SE  in  the  form  of  software 
maintenance  and  upgrade,  this  thesis  will  apply  the  Knowledge  Value  Added  (KVA) 
methodology  to  establish  the  baseline,  “As  Is,”  configuration  of  the  current  solutions  in 
AEGIS.  The  KVA  analysis  will  yield  the  ROI’s  and  the  current  models  for  the  approach 
to  software  maintenance  and  upgrade.  Based  on  the  assumptions  of  OA  design  for 
original  system  development,  new  approaches  to  distance  and  maintenance  and 
monitoring  will  be  explored  in  “To  Be”  solutions,  and  the  ROIs  will  be  estimated.  The 
“To  Be”  solutions  are  rooted  in  the  assumptions  of  MFOP  and  ARCI,  and  the  results 
indicate  that  these  solutions  yield  a  potential  improvement  of  720%  and  a  cost  saving  of 
$365,104.63  over  the  current  methodology  for  just  one  ship.  For  all  ships  using  AEGIS, 
ROI  improves  by  71,967%  with  a  cost  savings  of  $26,543,824.56.  The  conclusion  is  that 
OA  enables  extension  of  these  best  practice  approaches  to  AEGIS  maintenance  and 
upgrade  solutions. 
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I.  INTRODUCTION 


A.  PURPOSE 

The  need  has  always  existed  aboard  ship  to  maintain  operations  of  specific 
organic  assets  while  at  sea,  and  as  the  United  States  Navy  rapidly  advances  towards 
reduced  manning,  Integrated  Warfare  Systems  (IWS)  will  require  a  new  approach  to 
Sustaining  Engineering  (SE)  for  software  maintenance  and  upgrade  solutions  if  a  smaller 
crew  is  to  perform  at  the  same  caliber.  The  value  of  sustaining  engineering  and  creating 
more  efficient  software  upgrades  can  be  realized  by  increased  time  spent  on  mission 
efforts  and  decreased  time  spent  on  solving  IWS  anomalies.  The  more  efficient  system 
design  of  the  future  can  adapt  to  the  requirements  through  open  architecture  (OA),  and 
when  designers  use  an  OA  approach  it  enables  innovation  without  major  efforts  on  the 
part  of  the  ship’s  crew. 

A  previous  study  conducted  at  Naval  Postgraduate  School  by  Capt.  Joseph 
Uchytil  entitled,  “Assessing  the  Operational  Value  of  Situational  Awareness  of  AEGIS 
and  Ship  Self-Defense  System  (SSDS)  Platforms  through  the  Application  of  the 
Knowledge  Value  Added  (KVA)  Methodology,”  demonstrated  that  KVA  could  be  used 
to  estimate  the  performance  of  an  OA  implementation  in  terms  of  a  Return  on  Investment 
(ROI).  While  Capt.  UchytiTs  research  focused  on  benefits  derived  from  the  warfighter’s 
perspective,  the  purpose  of  this  research  is  to  generate  and  assess  the  impact  of  an  OA 
development  and  acquisition  approach  to  SE  in  IWS. 

By  extending  the  focus  of  this  OA  study  from  the  warfighter  to  the  acquirer  and 
system  developer  the  analysis  provides  a  more  complete  view  of  the  whole  system 
development  lifecycle  and  the  potential  of  OA  to  improve  the  perfonnance  of  the  cycle  in 
the  SE  processes.  Due  to  the  scope  of  this  study  it  is  more  likely  for  the  SE  portion  of  the 
life  cycle  to  achieve  the  highest  potential  productivity  which,  in  turn,  helps  to  make  sure 
they  exploit  the  benefits  of  OA  in  this  part  of  the  life  cycle  which  is  developer  and 
acquirer  intensive. 
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B. 


BACKGROUND 


The  benefit  of  open  architecture  from  the  developer  and  acquirer  point-of-view  is 
that  it  creates  greater  flexibility  by  introducing  additional  technologies  and  capabilities  to 
the  fleet  which  closed  systems  of  the  past  have  failed  to  introduce  after  procurement.  This 
is  because  closed  systems  are,  typically,  not  as  amenable  to  rapid  upgrades  as  open 
systems.  Current  systems  need  to  be  fluid  and  dynamic  to  respond  to  and  anticipate  the 
anomalies  encountered  on  ships.  Ergo,  it  is  no  longer  practical  for  software  maintenance 
support  and  upgrades  to  operate  within  closed  systems  because  they  are  difficult  and  slow 
to  upgrade,  have  limited  interoperability,  and  upgrades  must  be  done  with  the  same 
vendor.  Additionally,  it  is  hard  to  maintain  proprietary  systems  because  of  their 
interdependencies  in  code  and  software.  Due  to  lifecycle  time  and  cost  constraints,  OA, 
which  offers  faster  business  and  system  models  to  the  acquirer  and  developer  and 
independent  coding,  should  be  effectively  used  in  order  to  promote  the  future  view  of  a 
Navy  which  will  operate  within  the  Global  Infonnation  Grid  (GIG).  The  Global 
Infonnation  Grid  (GIG)  will  create  an  infonnative  and  integrated  environment  to  pass  the 
information.1  This  will  require  integration  of  several  parts,  and  it  will  require  those  parts 
to  be  fully  operational,  usable,  and  easily  upgraded  under  reduced  manning  and  joint 
operational  conditions.  One  of  the  enablers  of  GIG  will  be  the  effectiveness  of  open 
architecture  which  will  allow  for  faster  integration  when  applied  to  sustaining 
engineering  i.e.  the  software  maintenance  support  and  upgrade  process. 

C.  RESEARCH  OBJECTIVES 

The  objective  of  this  research  is  to  analyze  the  potential  benefits  of  open 
architecture  from  the  developers  and  acquirers  perspective  in  the  SE  process  for  the 
AEGIS  weapons  system.  This  will  be  achieved  through  the  KVA  approach  that  will 
provide  the  analytical  framework  to  assess  the  added  value  of  the  open  architecture 
approach  to  software  maintenance  support  and  upgrade  solutions  to  the  developer  and 
acquirer. 


1  Clark,  V.  2002. 
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The  KVA  approach  provides  the  static  ROI  analysis  models  which  serve  as  the 
input  for  Real  Options  which  will  be  conducted  in  further  research.  By  placing  both 
related  and  unrelated  elements  in  a  single  unit,  the  value  of  both  objects  can  be  compared 
in  one  lump  figure.  Please  refer  to  the  appendix  for  a  more  detailed  description  of  the 
KVA  methodology. 

D.  RESEARCH  QUESTIONS 

Since  the  measure  of  effectiveness  of  a  ship  in  the  fleet  does  not  rely  solely  on 
monetary  cost  savings,  but  rather  on  the  productivity  and  mission  accomplishment,  the 
knowledge  value-added  approach  can  be  applied  to  produce  a  return  on  investment  (ROI) 
that  will  serve  as  a  measure  of  productivity.  Possible  models  that  would  increase  the 
productivity  of  software  maintenance  support  and  upgrade  solutions  in  the  AEGIS 
weapons  system  can  be  explored  after  a  baseline  of  the  current  system  is  established. 

This  thesis  will  provide  “To-Be”  scenarios  for  using  an  open  architecture 
approach  to  meet  the  demands  of  the  future  Navy  with  regards  to  sustaining  engineering. 
The  secondary  research  will  explore  the  Department  of  Defense  and  Department  of  the 
Navy  initiatives  for  Open  Architecture,  Open  Architecture  Computing  Environments, 
Services  Oriented  Architecture  Solutions,  Distance  Support  Solutions,  and  current  “best 
practices”  examples  in  software  upgrades  in  military  environments. 

Our  analysis  will  answer  the  following  research  questions: 

*  Using  IBM’s  Component  Business  Modeling  (CBM),  what  are  the  areas 
of  highest  concern  and  cost  in  the  AEGIS  weapons  system  as  they  relate  to 
sustaining  engineering? 

*  What  are  the  “best  practice”  examples  of  sustaining  engineering  in  the 
commercial  or  military  environment  and  how  do  they  improve  the 
processes  of  system  development  and  acquisition  life-cycles?  What  are 
the  best  examples  of  SOA  and  distance  support  systems? 
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*  What  is  the  benefit  of  OA  in  the  process  of  sustaining  engineering?  We 
will  apply  the  KVA  methodology  to  the  areas  of  highest  concern  as 
identified  by  IBM  in  their  CBM  to  address  these  questions.  . 

*  What  overall  numerical  effect  does  OA  have  on  Sustaining  Engineering 
and  is  it  appreciable  and  low-risk  enough  to  system  development  and  the 
DoD  acquisition  lifecycle? 

This  research  will  provide  decision  makers  with  a  structured  analysis  of  employing  open 
architecture  to  improve  productivity  within  sustaining  engineering  and  software  upgrades 
for  Integrated  Warfare  Systems  (IWS). 

E.  METHODOLOGY 

We  will  employ  the  case  example  method  when  conducting  our  research.  We  will 
focus  on  exploring  various  avenues  of  improvement  associated  with  sustaining 
engineering,  specifically  the  software  maintenance  and  upgrade  processes  for  the  AEGIS 
system.  A  KVA  analysis  will  be  conducted  on  the  system  in  the  “As-Is”  configuration. 
This  will  serve  as  a  static  baseline  analysis  and  will  be  used  to  generate  the  To-Be” 
models.  The  KVA  and  follow-on  RO  analysis  in  future  research  will  then  be  conducted 
for  the  software  maintenance  and  upgrade  process  that  employs  an  OA  framework.  Both 
analyses  will  be  conducted  with  the  help  of  AEGIS  subject  matter  experts.  The  KVA 
analysis  associated  with  the  “As-Is”  and  the  “To-Be”  (employing  an  OA  framework) 
systems  will  produce  an  ROI  for  each  process  model.  The  ROI  associated  with  each 
process  model  will  be  compared  in  further  research  to  detennine  the  impact  of  OA  as  a 
viable  solution  to  improving  sustaining  engineering  and  the  software  maintenance  and 
upgrade  processes  associated  with  AEGIS.  The  follow  on  RO  analysis  will  identify 
options  associated  with  each  process  model  including  valuation  options,  cost  options  and 
risk  options.  The  results  will  be  compared  and  evaluated  and  the  benefit  of  OA  to 
sustaining  engineering  and  the  software  upgrade  processes  for  Integrated  Weapons 
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Systems  (IWS)  will  be  determined.  The  research  will  offer  recommendations  on  how  to 
improve  sustaining  engineering  and  the  software  upgrade  processes  in  the  context  of  OA 
systems. 

F.  SCOPE 

The  scope  of  this  thesis  will  be  to  specifically  prescribe  an  operational  value  to 
the  improvement  of  the  software  maintenance  and  upgrade  procedures  of  AEGIS  using 
OA.  This  research  highlights  the  inherent  value  of  knowledge  capital  in  a  system  by 
using  KVA  methodologies  and  it  emphasizes  the  need  to  introduce  OA  solutions  to  many 
more  sustaining  engineering  processes  aboard  ships  which  will  undergo  reduced  manning 
and  be  expected  to  achieve  “decision  superiority’  in  the  future. 

G.  THESIS  ORGANIZATION 

The  chapter  organization  will  be:  Chapter  I  will  give  a  general  overview  of  the 
purpose  and  intended  methods  and  scope  of  the  thesis.  Chapter  II  will  provide  secondary 
research  and  background  infonnation  on  open  architecture  aims,  OACE,  SOA,  distance 
support  solutions,  and  best  practice  examples.  Chapter  III  will  consist  of  the  KVA 
analysis  giving  the  resulting  figures  and  charts.  Chapter  IV  will  discuss  the  results  from 
the  KVA  analysis  and  the  implications  of  the  current  “As-Is”  state  of  AEGIS 
maintenance  and  then  explore  the  “To-Be”  results.  Chapter  V  will  present  conclusions 
from  the  research  that  was  conducted.  Chapter  VI  will  recommend  further  research  that 
can  be  conducted  to  continue  the  process  of  refining  sustaining  engineering  and  software 
upgrades  within  the  context  of  open  architecture  in  the  fleet. 
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II.  LITERATURE  REVIEW:  SOFTWARE  UPGRADES  AND 
MAINTENANCE  SOLUTIONS  IN  THE  OPEN  ARCHITECTURE 

ENVIRONMENT 


A.  CBM  THEORY  AND  SUSTAINING  ENGINEERING 

The  driving  factor  that  has  promoted  the  need  for  improvement  of  Sustaining 
Engineering  (SE)  in  software  maintenance  and  upgrade  is  the  Component  Business 
Model  (CBM)  process  conducted  by  IBM  in  June  of  2006.  Component  Business 
Modeling  is  an  “IBM-developed  technique  for  representing  an  enterprise  as  non¬ 
overlapping  business  components  in  order  to  identify  opportunities  for  innovation  and/or 
improvement.”2  In  a  CBM,  a  business  component  is  a  group  of  resources,  people,  and 
technology  that  have  the  necessary  information  to  deliver  value  from  functional 
performance.3  The  final  result  of  the  grouping  of  business  components  is  visually 
represented  in  a  component  business  map  which  hones  in  on  the  essential  foundational 
blocks  of  the  organization.  Table  1  is  the  final  component  business  map  for  the 
breakdown  of  AEGIS  in  Program  Executive  Office  Integrated  Warfare  Systems  (PEO 
IWS)  for  Fiscal  Year  2007. 


2  Pavlick,  Tim  2005;  7. 

3  Ibid.,  7. 
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4  Shannon,  James  2006;  14. 
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“Hot  components,”  or  components  that  are  worth  further  examination,  are 
represented  by  a  star.  They  are  identified  as  hot  components  based  on  attributes  selected 
as  important  to  the  organization  being  assisted  through  the  analysis.  In  the  case  of  this 
CBM  effort,  there  were  three  criteria  selected:  investment  of  total  budget  (green),  number 
of  efforts  required  for  the  task  (yellow),  and  color  of  money  (orange).5  For  the  sustaining 
engineering  category,  it  has  a  medium  percentage  of  the  PEO  IWS  budget,  a  high  number 
of  efforts  (greater  than  six),  and  two  colors  of  money  involved.  The  colors  of  money,  or 
the  money  which  is  procured  and  used  for  specific  acquisitions,  are  in  the  areas  of 
Operation  and  Maintenance  (OMN)  and  Ship  Building  and  Conversion  (SCN).  The 
horizontal  axis  represents  a  key  competency,  or  one  which  requires  similar  skills  and 
capabilities  while  the  vertical  axis  represents  accountability  levels.  SE  is  “System 
Sustainment  and  Disposal”  competency,  in  which  the  “executing,”  branch,  the  branch 
that  does  the  work,  is  accountable. 

In  addition  to  SE  being  identified  as  a  “hot  component,”  the  Operations  and 
Support  (O&S)  phase  of  a  system’s  life  cycle  is  often  represented  to  incur  80%  of  the 
total  life  cycle  costs  of  a  system.  According  to  an  article  published  in  Program  Managers 
Magazine,  weapons  system  sustainment  consumes  about  80  percent  of  logistics 
resources,  or  approximately  64B  per  year.”6  With  such  a  large  factor  of  the  total  life 
cycle  costs  being  focused  in  this  life  cycle  phase,  along  with  the  CBM  results,  it  is 
reasonable  to  examine  Sustaining  Engineering  for  ways  to  make  it  more  efficient.  IBM 
also  anticipates  that  a  large  cost  component  within  O&S  is  SE.7  In  table  2  this  is  evident. 
In  fact,  this  is  the  only  starred  area  on  the  CBM  diagram  in  which  all  colors  of  money 
will  increase  spending.  KVA  will  seek  to  give  decision  makers  a  tool-set  for  making  the 
vast  amount  of  spending  on  SE  more  efficient  through  the  use  of  OA. 


5  Shannon,  James  2006;  11. 

6  Kratz,  Louis  A.  2002;  2. 

7  Shannon,  James  2006;  16. 
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Table  2.  Sustaining  Engineering  Color  of  Money  Cost  Increase  8 


B.  OPEN  ARCHITECTURE  ENVIRONMENT  AND  TENETS 

To  achieve  efficiency  in  software  upgrade  and  maintenance  it  is  necessary  to 
eliminate  current  inadequacies  in  implementing  open  architecture.  Department  of 
Defense  systems,  according  to  a  report  release  in  2006  by  the  Government  Accountability 
Office,  continue  to  lag  behind  in  interoperability,  even  though  the  Program  Executive 
Office,  Integrated  Warfare  Systems  (PEO  IWS)  was  established  in  2002  and  was  in 
charge  of  executing  OA.8 9  Perhaps  this  is  a  result  of  the  well  known  and  frequently 
addressed  security  concerns  involved  in  implementing  OA  in  weapons  systems,  such  as 
malicious  code,  computer  viruses  or  system  latency;  however,  the  civilian  sector  grasped 
the  concept  quickly  and  they  also  have  a  need  for  security.  Even  banks,  for  the  most  part, 
operate  with  the  framework  of  OA.  Some  banks  even  operate  with  Service  Oriented 
Architecture  which  will  be  defined  and  discussed  at  the  conclusion  of  the  literature 
review. 

To  implement  OA,  it  is  necessary  to  understand  some  basic  concepts.  In  a  general 
sense,  OA  is  realized  through  rapid  change  and  fluid  upgrades  and  solutions.  According 
to  the  Deputy  Chief  of  Naval  Operations,  the  requirements  for  OA  implementation  are  as 
follows:  modular  design  and  design  disclosure,  reusable  application  software, 


8  Shannon,  James  2006;  16. 

9  Government  Accountability  Office  2006. 
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interoperable  joint  warfighting  applications  and  secure  information  exchange,  life  cycle 
affordability,  encouraging  competition  and  collaboration,  scalability  and  portability.10 

1.  Modular  Design  and  Design  Disclosure 

Modularity  is  the  concept  of  decomposing  a  system  into  transparent 
subcomponents.11  These  subcomponents  are  operable  without  relying  on  another  aspect 
of  the  system;  hence,  they  can  rapidly  change  and  allow  for  interactions  with  numerous 
systems.  The  underlying  goal  of  decomposition,  in  the  case  of  modularity,  is  to  allow  for 
the  independent  upgrade  of  each  of  the  smallest  subcomponents  while  leaving  the 
complete  system  operable.  With  modular  design  and  design  disclosure,  multiple 
competitors  can  participate  and  innovation  flourishes  as  each  subcomponent  is 
independently  tried  and  tested. 

2.  Reusable  Application  Software 

Reuse  allows  a  system  to  use  the  same  components  and  code  that  have  been  used 
across  other  platforms.12  In  the  case  of  application  software,  a  database  of  segments  of 
code  that  worked  for  the  tracking  device  of  one  platform  can  be  shared  when  creating 
other  tracking  devices.  This  would  be  a  database  that  would  be  continually  updated  with 
components  and  segments  of  code  that  could  have  potential  use  in  other  areas.  These 
components  can  be  used  interchangeably  with  other  components  without  affecting  the 
system  in  its  entirety.  This  idea  is  revolutionary  for  coding  and  software  upgrade  in  much 
the  same  way  that  “interchangeable  parts”  revolutionized  the  assembly  lines  of  the  1920’s 
with  increased  output  and  increased  revenue.  Disclosure  of  the  design  of  application 
software  would  also  be  necessary  for  evolutionary  improvement  in  future  upgrades.13 


10  Chief  of  Naval  Operations  2005. 

11  Coronado  Mondragon,  Christian  E.  2006;  247. 

12  Chief  of  Naval  Operations  2005. 

13  Ibid. 
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3.  Interoperable  Joint  Warfighting  Applications  and  Secure  Information 
Exchange 

This  particular  tenet  ensures  that  across  a  wide  variety  of  systems  the  same 
information  and  applications  can  be  shared.  It  involves  commonality  of  services, 
warfighting  applications,  and  infonnation  assurance  and  requires  these  commonalities  to 
be  essential  for  the  basic  design  elements  of  any  new  system.14 

4.  Life  Cycle  Affordability 

This  tenet  includes  all  phases  of  the  life  cycle  from  design  and  requirements 
gathering  to  delivery  and  testing.  Since  the  primary  concern  of  this  thesis  is  the  sustaining 
engineering  portion,  and  since  SE  is  such  a  large  chuck  of  the  life  cycle  costs,  the  results 
could  directly  benefit  the  implementation  of  OA  with  respect  to  life  cycle  affordability. 

5.  Encouraging  Competition  and  Collaboration 

OA  naturally  encourages  competition  and  collaboration  because  unlike  closed 
systems  many  different  systems  can  be  integrated  to  complete  upgrades  or  create  a  new 
system.  That  is  not  to  say  that  proprietary  systems  did  not  contain  many  different  parts 
that  required  different  companies  to  collaborate  but  they  were  less  likely  to  constantly 
create  an  environment  of  competition  and  innovation  because  some  of  the  contracts  were 
sole-source.  Sole  source  contracts  being  those  which  restrict  Full  Open  Competition  as  it 
is  a  non-competitive  procurement  process  in  which  solicitation  is  only  with  one  source,15 

6.  Scalability 

Scalability  encompasses  the  introduction  of  new  functionalities  into  a  system 
without  procuring  a  whole  new  system  to  do  the  same  job.  An  example  of  scalability  is 
the  method  of  increasing  bandwidth  during  the  holiday  season  to  allow  for  faster 
transactions  during  a  season  of  heightened  traffic. 


14  Chief  of  Naval  Operations  2005. 

15  Department  of  Defense  5000.1  2003. 
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7.  Portability 


Portability  is  the  ability  of  the  software  or  hardware  and  the  users  to  easily 
integrate  into  different  platforms.  It  requires  source  code  to  make  transitions  between 
hardware  and  software  and  requires  the  switch  to  be  rapidly  and  smoothly  accomplished. 

C.  LEGACY  SYSTEMS  AND  THEIR  EFFECT  ON  SOFTWARE  UPGRADES 

AND  MAINTENANCE 

For  the  most  part,  closed  systems  of  the  past  contain  software  which  is  designed 
for  the  purpose  of  supporting  the  computing  hardware.  When  proprietary  systems  do 
need  upgrades,  computer  code  must  change  as  well,  but  their  unique  design  sometimes 
makes  software  upgrades  financially  and  technically  prohibitive.  Programs  such  as  these 
delay  the  time  to  introduce  new  technologies  to  the  fleet  and  increase  the  total  life-cycle 
cost  of  the  system.  Table  1,  shown  below,  lists  the  contrasting  characteristics  between 
closed  systems  and  open  systems.  Most  important  to  this  research  are  three  points: 

*  That  expansion  and  upgrade  of  closed  systems  requires  more  time,  effort, 
and  money  than  the  open  systems 

*  That  closed  systems  are  less  adaptable  to  changes  in  threats  and  new 
technologies. 

*  That  closed  systems  are  focused  mostly  on  development  cost  and  meeting 
the  present  mission  while  open  systems  focus  on  the  total  costs  of 
ownership,  sustainment,  and  growth  of  the  system. 
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Closed  System  Characteristics 

Open  System  Characteristics 

Use  of  closely  held,  private  interfaces, 
languages,  data  fonnats  and  protocols 
(government  or  vendor  unique  standards) 

Use  of  publicly  available  and  widely  used 
interfaces,  languages,  data  formats  and 
protocols 

Critical  importance  is  given  to  unique 
design  and  implementation 

Critical  importance  is  given  to  interfaces 
management,  and  widely  used  conventions 

Less  emphasis  on  modularity 

Heavy  emphasis  on  modularity 

Vendor  and  technology  dependency 

Vendor  and  technology  independence 

Minimization  of  the  number  of 
implementations 

Minimization  of  the  number  of  types  of 
interfaces 

Difficult  and  more  costly  integration 

High  degree  of  portability,  connectivity, 
interoperability,  and  scalability 

Use  of  sole-source  vendor 

Use  of  multiple  vendors 

Expansion  and  upgrading  usually  requires 
considerable  time,  money  and  effort 

Easier,  quicker  and  less  expensive 
expansion  and  upgrading 

Higher  total  ownership  cost 

Lower  total  ownership  cost 

Slower  and  more  costly  technology  to 
transfer 

Technology  transfer  is  faster  and  less 
costly 

Components,  interfaces,  standards,  and 
implementations  are  selected  sequentially 

Components,  interfaces,  standards,  and 
implementations  are  selected  interactively 

Systems  with  shorter  life  expectancy 

Systems  with  longer  life  expectancy 

Use  of  individual  company  preferences  to 
set  and  maintain  specifications 

Use  of  group  consensus  process  to 
maintain  interface  specifications 

Less  adaptable  to  change  in  threats  and 
technologies 

More  adaptable  to  evolving  threats  and 
technologies 

Focusing  mostly  on  development  cost  and 
meeting  present  mission 

Focusing  on  total  costs  of  ownership, 
sustainment  and  growth 

User  as  the  producer  of  system 

User  as  the  consumer  of  components 

Rigid  and  slow  system  of  influence  and 
control 

Real  time  and  cybernetic  system  of 
influence  and  control 

Adversarial  relationship  with  prime 
contractors/supplier/vendors 

Symbiotic  relationship  with  prime 
contractors/suppliers/vendors 

Mostly  confined  to  traditional  suppliers 

Non-traditional  suppliers  can  compete 

Simple  conformance  testing 

Very  challenging  conformance  testing 

Table  3.  Open  Systems  v.  Closed  Systems16 


16  Azani,  C.  2001;  1. 
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Legacy  systems  also  have  a  specific  computational  power  limitation.  Systems  like 
the  AEGIS  6Ph3  radar  processing  has  software,  which  relies  on  the  military  standard 
computer,  UYK-43,  which  was  sole-source  contracted  to  Lockheed  Martin  in  1980. 17 
Such  systems  cannot  keep  up  with  the  steadily  increasing  computational  power  in  the 
commercial  sector.  The  negative  effect  on  current,  closed  systems  is  magnified  because 
they  are  fast  becoming  obsolete  while  the  benefits  of  OA  are  not  realized.  This  research 
seeks  to  shift  the  focus  of  the  AEGIS  system  software  upgrades  to  increase  the  overall 
value  of  the  system  both  monetarily,  operationally,  and  from  a  user’s  perspective  by 
proving  the  worth  of  the  implementation  of  open  architecture  in  software  maintenance 
and  development  where  it  is  most  amenable. 

D.  OPEN  ARCHITECTURE  COMPUTING  ENVIRONMENT 

As  stated  previously,  the  high  costs  of  computer  program  maintenance  and 
development  are  attributed  to  obsolescence,  frequently  needed  changes,  and  proprietary 
systems  which  contain  software  applications  that  are  closely  linked  to  the  backbone  of 
system  operation.  Maintenance  and  development  of  such  software  could  adversely  affect 
the  system  as  a  whole,  thus  making  it  less  amenable  to  any  type  of  change  which, 
hypothetically,  is  avoidable  in  an  age  with  rapid  development  of  commercial  off-the-shelf 
(COTS)  technology. 

According  to  the  directive  for  Network  Centric  Warfare,  the  open  architecture 
concept  will  be  applied  not  only  to  hardware  but  also  to  software  and  the  computing 
environment.  Open  Architecture  Computing  Environment  (OACE)  is,  at  its  essence,  the 
application  of  open  architecture  to  computing  systems  so  that  over  the  life  of  the  platform 
changes  can  be  made  with  commercial  technologies  that  will  rapidly  meet  the  changing 
demands  of  reduced  manning  source.  Closed  systems  of  the  past  reduced  the  ability  of 
developers  to  modernize  the  system  and  to  provide  maintenance  solutions  for  underlying 
problems.  They  also  robbed  the  acquisition  field  of  competitive  contracts,  as  their  field  of 
suppliers  was  limited.  In  existing  proprietary  systems,  obsolescence  and  the  inability  to 
introduce  upgrades  has  decreased  the  overall  value  of  acquiring  the  system.  Vice 


17  FAS  Military  Network  Analysis,  1998. 
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Admiral  John  Nathan  said  to  the  House  Armed  Services  Committee  in  2003,  "By 
pursuing  an  Open  Architecture  and  an  Open  Architecture  Computing  Environment  based 
on  mainstream  COTS  technologies,  systems  and  standards,  we  can  avoid  the  high  cost  of 
maintaining  and  upgrading  multiple  legacy  computing  systems  that  quickly  become 
obsolete  and  are  not  responsive  to  changes  in  war-fighting  requirements."18  Additionally, 
DoD  directive  5000.1  states,  “a  modular,  open-systems  approach  shall  be  employed 
where  feasible,"  and  a  memorandum  in  April  2004  from  the  Under  Secretary  of  Defense 
for  Acquisition,  Technology,  and  Logistics,  expands  on  that  language  by  requiring  that 
all  programs  are  subject  to  a  milestone  review  brief  their  modular  open-systems 
approach.19 

1.  OACE  Shift 

The  move  to  the  OACE  is  designed  to  be  incremental  and  is  currently 
implemented  in  four  catagories  as  Table  2  below  demonstrates. 


18  Nathan,  John  2003. 

19  Department  of  Defense  5000.1,  2003. 
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Table  4.  OACE  Incremental  Compliance20 


2.  OACE  Category  3 

Open  Architecture  Computing  Environment  category  3  describes  complete 
compliance  with  all  OACE  standards  to  include,  physical  media,  networks,  operating 
systems,  middleware  and  programming  languages.  This  is  critical  in  reuse  of  components 
and  allows  for  the  interoperability  between  different  computing  infrastructures.  The  goal 
for  the  full  integration  of  Category  3  is  the  year  2008,  with  the  main  component  being  the 


20  Department  of  Defense,  NAVSEA,  PEO  IWS  2004. 
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standard  middleware.21  Middleware  is  the  use  of  software  which  allows  interoperability 
between  two  different  closed  systems.  Rather  than  proprietary  based  middleware, 
standards-based  solutions  will  be  used  instead  to  meet  the  ever-changing  requirements  of 
the  system.  Standards-based  solutions  are  those  which  meet  the  industry  regulated  norm, 
such  as  the  “user-friendly”  norm  which  Microsoft  created  when  they  released  windows 
on  an  international  level. 

E.  SECURITY  QUESTIONS 

One  central  concern  of  the  Department  of  Defense  with  regards  to  the  shift  to  OA 
and  OACE  is  the  need  to  maintain  security  in  military  systems.  Some  have  speculated 
that  to  let  open  architecture  be  the  prevailing  architecture  for  fleet  releasable  software 
maintenance  and  upgrade  would  be  to  let  the  proverbial  "wolf  in  sheep's  clothing" 
penetrate  our  defenses.  These  concerns  stem  from  the  fear  of  malicious  code  causing  a 
whole  defense  system  to  malfunction.  Supporters  of  open  architecture  state  that  because 
newer  systems  are  so  open  to  review  by  so  many  different  sources  that  the  possibility  of 
malicious  code  passing  under  the  eyes  of  so  many  is  slim. 

F.  SERVICE  ORIENTED  ARCHITECTURE  (SOA)  SOLUTIONS 

Service  oriented  architecture  is  a  permutation  of  OA  in  that  it  is  the  ultimate  in  the 
OA  Tenets  of  “modularity”  and  “reuse.”  SOA  seeks  to  combine  many  different  services 
that  communicate  through  XML  messages  across  a  common  web  so  that  they  can  be 
interchangeably  used  to  complete  a  task.  It  makes  software  based  changes  rather  than 
hardware  based  changes.  Table  3  is  a  before  and  after  example  of  the  implementation  of 
SOA,  and  it  has  both  pros  and  cons. 


21  Department  of  Defense,  NAVSEA,  PEO  IWS  2004. 
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Table  5.  Before  &  After  SOA22 


1.  SOA  Benefits 

With  the  introduction  of  SOA  any  service  that  requires  examining,  such  as  a 
broken  pipe  or  a  computer  that  has  “crashed”  can  be  evaluated  by  meshing  whichever 
portions  of  an  organization  that  need  to  be  meshed  together  to  address  the  specific 
problem  at  the  time  of  an  anomaly.  Some  other  benefits  include  a  higher  potential  ROI, 
visibility  or  enterprise  level  business  processes,  reduced  redundancy  and  ease  of 
interoperability  internally  and  between  third  parties  23  SOA  could  be  extremely  beneficial 
in  this  case  with  organizational  “stove-pipes”  and  bridging  legacy  systems  that  otherwise 
would  be  non-interoperable  in  the  environment. 


22  U.S.  Army  PEO  IES  2006. 

23  Ibid. 
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2. 


SOA  Drawbacks 


There  are  several  drawbacks  to  SOA.  If  there  is  a  large  volume  of  transactions 
within  the  system,  for  instance,  an  online  bank,  then  SOA  would  require  a  massive 
amount  of  time  and  dedicated  resources  to  fully  realize  its  potential.  Additionally, 
security  is  as  much  of  an  issue  in  SOA  as  it  is  in  OA.  At  times  it  may  be  easier  to  have 
tightly  group  interfaces  with  a  history  of  collaboration,  rather  than  loose  interfaces.  This 
is  especially  sensitive  where  AEGIS  is  concerned  because  if  the  interface  is  not  tightly 
coupled,  then  any  latency  present  could  cause  enough  delay  to  give  an  enemy  superiority 
over  our  assets.  As  an  air  defense  platform  any  increase  in  latency  would  be 
unacceptable. 

G.  DISTANCE  SUPPORT  MAINTENANCE  SOLUTIONS 

According  to  2006  Distance  Support  Policy  released  by  the  Chief  of  Naval 
Operations,  distance  support  is  rapidly  becoming  “the  Fleet’s  principal  web-based 
readiness  enabler.”24  At  a  minimum,  the  current  distance  support  system,  “combines 
people,  processes,  and  technology  into  a  collaborative  infrastructure  without  regard  to 
geographic  location.”25  In  other  words,  ships  can  be  underway  for  several  months  and 
communicate  with  shore  based  sites  to  fix  software  and  hardware  problems  that  occur  on 
board  and,  hopefully,  resolve  them  without  pulling  into  a  foreign  port  or  returning  to  the 
shipyards. 

The  future  of  distance  support  will  also  include  shore  based  monitoring  of 
systems,  in  much  the  same  way  that  cars  sold  in  2006  and  2007  can  communicate  with 
central  databases  and  give  a  report  of  their  technical  status  which  is  then  emailed  to  the 
owner  of  the  vehicle.  This  fonn  of  distance  support,  called  remote  monitoring  and 
notification,  in  a  possible  form  of  procedure  for  shipboard  operations,  is  displayed  in 
Table  4  below. 


24  Chief  of  Naval  Operations  2006. 

25  Ibid. 
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Table  6.  Remote  Monitoring  and  Notification 

From  the  table,  the  shipboard  information  is  constantly  monitored  in  the 
“data/information  acquired  phase”  which  then  relays  the  information  to  the  shore-side 
server  for  diagnostics  and  assessments  of  trends  and  material  readiness.  If  there  is  a 
problem  with  the  systems  maintenance  and  risk  recommendations  are  made,  documented, 
and  then  analyzed  in  metrics;  the  ship  is  notified.  If  there  is  no  detected  issue  then  the 
monitoring  cycle  will  repeat  and  send  a  clean  report  to  the  ship. 

When  distance  support  is  used  correctly  it  complements  the  tenets  of  OA  quite 
effectively,  if  upgrades  and  the  necessary  changes  can  be  made  to  an  open  system.  It 
allows  for  modularity  and  design  reuse  that  can  be  distantly  monitored  and  repaired 
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rather  than  requiring  a  visit  to  port  for  the  problem  to  be  fixed.  Additionally,  in  the  figure 
above,  remote  monitoring  can  provide  automation  in  the  process  of  upgrades  and  repairs, 
and  automation  in  most  cases,  leads  to  a  decrease  in  number  of  employees,  hence,  an 
increase  in  a  systems’  Return  on  Investment 

H.  BEST  PRACTICES 

1.  Distance  Support 

a.  Maintenance  Free  Operating  Period  (MFOP)  and  Acoustic 
Rapid  Commercial  Off-the-shelf  (COTS)  Insertion  (ARCI) 

In  2005,  Naval  Sea  Systems  Command  (NAVSEA)  completed  a  pilot 
program  to  test  the  feasibility  of  a  Maintenance  Free  Operating  Period  (MFOP)  on  the 
Acoustic  Rapid  COTS  Insertion  (ACRI)  System.  The  ARCI  system  was  designed  to 
replace  the  AN/BSY-1  and  the  AN/BQQ-5  on  the  fleet’s  in-service  submarines 
(688/688I/Trident/Seawolf).26  ARCI  was  a  success  in  its  own  right,  in  that  it  effectively 
demonstrated  the  use  of  OA  with  COTS  technology  on  a  large  scale  in  the  fleet  and 
allowed  for  technology  insertion  and  refreshment.27 

The  ARCI  MFOP  program  was  conducted  over  a  one  year  time  span  and  it 
tested  the  use  COTS  technology  and  the  COTS  support  provided  to  design  ARCI  in  such 
a  way  to  enable  MFOP.  Four  platforms  participated  in  the  testing  and  over  the  course  of 
one  year  no  maintenance  was  required  in  any  of  the  four.  One  resulting  benefit  of  this 
test,  for  the  purposes  of  this  research,  was  that  they  implemented  distance  support 
capabilities  into  the  ARCI  system  before  they  conducted  the  test.28  Most  particularly,  the 
following  results  are  applicable  for  the  formulation  of  “To  Be”  models  for  AEGIS 
software  maintenance  and  upgrade: 


2®  Lockheed  Martin  2005. 

27  Ibid. 

NAVSEA  Surface  Warfare  Logistics  and  Maintenance  2005. 
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A  database  of  maintenance  related  data  was  built  into  the  ARCI  system 
which  provides  the  capability  to  perform  statistical  analysis  of  system 
performance  and  improve  availability.  An  availability  correlation  function 
was  developed  to  monitor  system  parameters  and  make  recovery 
recommendations  to  system  operators...  An  additional  benefit  of  the  MFOP 
Pilot  Program  was  to  develop  and  implement  functionality  in  the  ARCI  system 
which  further  enables  the  system  to  be  supported  via  Distance  Support 
initiatives.29 

Using  the  advances  outlined  above,  the  “To-Be”  models  were  formulated  and  the  basis 
for  those  changes  was  grounded  in  research  that  has  proven  its  MFOP  reliability  over  the 
course  of  an  entire  year. 


29  NAVSEA  Surface  Warfare  Logistics  and  Maintenance  2005. 


23 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


24 


III.  PROCESS  DIAGRAMS  AND  “AS  IS”  MODELS 


A.  INTRODUCTION 

Program  Executive  Office,  Integrated  Warfare  Systems  (PEO  IWS),  with  its 
creation  in  2002,  began  an  initiative  to  implement  open  architecture  (OA)  throughout  the 
Navy’s  Integrated  Warfare  Systems.  One  of  the  current  initiatives  is  to  implement  OA 
into  the  sustaining  engineering  process  associated  with  the  AEGIS  system. 

Sustaining  engineering  in  the  AEGIS  system  was  identified  as  an  area  of  concern 
by  a  Component  Business  Model  (CBM)  analysis  completed  by  IBM.  The  AEGIS 
software  maintenance  and  upgrade  process  was  further  identified  by  the  research  team  as 
an  area  where  the  most  improvements  could  be  made  by  implementing  an  open 
architecture  framework.  In  order  to  accomplish  the  implementation  of  open  architecture 
into  the  AEGIS  software  maintenance  and  upgrade  process,  metrics  must  be  determined 
to  discover  which  areas  of  the  software  maintenance  and  upgrade  process  would  be  the 
best  candidates  for  open  architecture. 

This  proof  of  concept/case  study  will  utilize  information  gathered  from  subject 
matter  experts  (SME’s)  in  both  the  Surface  Warfare  Fleet  and  the  Naval  Surface  Warfare 
Center  (NSWC).  The  process  infonnation  gathered  from  these  subject  matter  experts 
will  be  utilized  to  provide  a  Return  on  Investment  (ROI)  analysis  using  the  Knowledge 
Value  Added  (KVA)  methodology.  This  will  be  an  analysis  of  the  “As  Is”  system 
configuration  or  the  baseline  case.  The  ROI  discovered  through  the  KVA  methodology 
will  be  analyzed  to  determine  if  open  architecture  could  improve  the  sustaining 
engineering  process.  A  Real  Options  (RO)  analysis  will  be  conducted  in  future  research 
on  the  “To  Be”  software  maintenance  and  upgrade  process  model  in  order  to  provide 
PEO  IWS  with  options  and  risks  for  future  courses  of  action. 
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B.  RESEARCH  QUESTIONS 


Measure  of  effectiveness  (MOE)  for  open  architecture  systems  have  been 
accurately  derived  through  the  Knowledge  Value  Added  Methodology.  This  proof  of 
concept  was  conducted  in  previous  research  by  Capt.  Joseph  Uchytil,  USMC  in  his  thesis, 
“Assessing  the  Operational  Value  of  Situational  Awareness  for  AEGIS  and  Ship  Self 
Defense  System  (SSDS)  Platforms  through  the  Application  of  the  Knowledge  Value 
Added  (KVA)  Methodology.”  This  thesis  is  intended  to  draw  on  Capt.  UchytiTs  proof  of 
concept  in  order  to  answer  the  following  research  questions: 

*  Using  IBM’s  Component  Business  Modeling  (CBM),  what  are  the  areas 
of  highest  concern  and  cost  in  the  AEGIS  weapons  system  as  they  relate  to  sustaining 
engineering? 

*  What  are  the  “best  practice”  examples  of  sustaining  engineering  in  the 
commercial  or  military  environment  and  how  do  they  improve  the  processes  of  system 
development  and  acquisition  life-cycles?  What  are  the  best  examples  of  “design  for 
maintenance”  systems? 

*  What  is  the  benefit  of  OA  in  the  process  of  sustaining  engineering?  We 
will  apply  the  KVA  methodology  to  the  areas  of  highest  concern  as  identified  by  IBM  in 
their  CBM  to  address  these  questions.  . 

*  What  overall  numerical  effect  does  OA  have  on  Sustaining  Engineering 
and  is  it  appreciable  and  low-risk  enough  to  system  development  and  the  DOD 
acquisition  lifecycle? 

C.  ANALYSIS  AND  DATA  COLLECTION 

1.  The  Software  Maintenance  and  Upgrade  Process 

The  AEGIS  software  maintenance  and  upgrade  process  is  a  very  complex  method 

encapsulating  a  large  number  of  processes  in  four  main  phases.  The  phases  are  the 

requirements  definition  phase,  the  design  phase,  the  test  phase,  and  the  implementation  or 

installation  phase.  These  are  the  basic  phases  in  any  software  maintenance/upgrade 
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process  whether  commercial,  government,  or  non-profit.  Depending  on  the  type  of 
upgrade  required,  the  severity  of  the  problem  it  is  intended  to  fix,  and  the  timeliness  in 
which  it  is  installed,  the  process  can  be  slightly  tailored  to  produce  more  immediate 
results. 

One  example  of  this  process  being  tailored  to  fit  a  certain  fleet  need  would  be 
when  an  Emergent  Update  is  required  was  discussed.  The  Emergent  Update  process  is 
utilized  for  problems  with  software  that  are  considered  “showstoppers”  or  a  Priority  1A 
problem.  Pending  approval  by  both  the  project  manager  and  the  customer,  the  software 
maintenance  and  update  process  will  be  tailored  to  address  only  the  Priority  1A  process 
and  will  be  completed  in  approximately  one  month.  Emergent  Updates  happen  on  rare 
occasions  and  approximately  95%  of  AEGIS  software  upgrades  go  through  the  entire 
software  maintenance  and  upgrade  process. 

The  entire  AEGIS  software  upgrade  life  cycle  is  intended  to  take  eighteen 
months,  but  typically  takes  closer  to  twenty  four  months  due  to  problems  that  are  found 
during  the  testing  phase  or  failure  of  certifications.  This  software  maintenance  and 
upgrade  process  involves  many  sub  processes  in  each  one  of  its  main  processes.  These 
sub  processes  may,  or  may  not,  have  a  bearing  on  the  rest  of  the  processes  in  the  software 
maintenance  and  upgrade  process.  The  fact  that  some  of  these  sub  processes  may  be  able 
to  function  in  a  stand  alone  capacity  makes  the  analysis  of  the  software  maintenance  and 
upgrade  process  very  difficult. 

2.  Data  Collection  Challenge 

Due  to  the  complex  nature  of  the  software  maintenance  and  upgrade  process  and 
the  large  number  of  people  involved,  collecting  accurate  data  to  be  used  by  the  KVA 
methodology  proved  to  be  a  challenge.  There  were  only  a  few  SMEs  who  understood  the 
complexity  of  the  process.  Also,  outputs  and  learning  time  associated  with  each  process 
and  sub  process  are  not  documented.  This  is  coupled  with  the  confusion  that  occurs 
between  learning  time  and  time  spent  in  a  Navy  training  course  to  leam  the  job.  The 
Navy  training  courses  are  often  of  a  uniform  length  of  time,  no  matter  the  complexity  of 
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job  and  subject  matter  experts  often  confuse  these  training  times  with  actual  learning 
time.  This  leads  to  a  slow  data  collection  process  because  the  data  needed  for  the  KVA 
analysis  is  not  readily  available.  There  is  also  a  need  to  separate  the  time  spent  in  a  Navy 
training  course  or  school  learning  the  specific  process  and  time  spent  learning  other 
skills.  Due  to  these  concerns,  data  collected  for  this  analysis  was  collected  through 
conversations  and  surveys  given  to  Subject  Matter  Experts  (SME’s).  The  data  was  then 
aggregated  to  capture  the  AEGIS  software  maintenance  and  upgrade  process  as  a  whole. 

D.  DEFINING  THE  AEGIS  SOFTWARE  MAINTENANCE  AND  UPDATE 

PROCESS 

1.  AEGIS  Software  Maintenance  and  Update  Process  Overview 

The  AEGIS  software  maintenance  and  update  process  takes  place  in  two  primary 
areas;  on  ship  and  off  ship.  The  on  ship  portion  of  the  process  takes  place  aboard  an 
AEGIS  equipped  U.S.  Naval  vessel  and  is  conducted  by  Surface  Warfare  fleet  personnel 
and  various  support  personnel  including  contractors.  The  on  ship  portion  of  the  software 
maintenance  and  update  process  deals  with  identifying  problems  that  were  not  found  in 
the  testing  phase  of  the  process  and  also  deals  with  installation  and  on  ship  testing  of  the 
fielded  AEGIS  software  update.  The  off  ship  portion  of  the  process  takes  place  at  Naval 
Surface  Warfare  Center,  Dahlgren,  VA.  This  primarily  deals  with  the  requirements 
definition  phase,  the  design  phase  and  the  testing  phase. 

2.  Defined  AEGIS  Software  Maintenance  and  Update  Processes 

Since  the  AEGIS  software  maintenance  and  update  process  is  a  complex  process 
involving  may  processes  and  sub  processes,  it  was  necessary  to  breakdown  the  process 
into  each  of  its  individual  processes  and  sub  processes.  This  breakdown  allows  for  a 
more  detailed  analysis  of  each  process  contained  within  the  software  maintenance  and 
update  process. 

The  aggregate  processes  associated  with  the  AEGIS  software  maintenance  and 
update  process  are  depicted  in  Table  7.  These  processes  were  developed  as  a  result  of 
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communication  with  several  Subject  Matter  Experts.  Although  the  process  could  differ 
for  high  priority  updates,  such  as  an  emergent  update,  the  process  depicted  encompasses 
almost  all  AEGIS  software  maintenance  and  updates.  Some  sub  processes  were  captured 
within  their  larger  process  to  provide  a  level  of  decomposition  that  was  sufficient  to 
produce  accurate  results  for  the  KVA  methodology. 
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AEGIS  WEAPONS  SYSTEM 
SOFTWARE  UPDATE 
PROCESS 

2/24/2007 


Table  7.  AEGIS  software  maintenance  and  update  process  (Aggregate  level) 
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E. 


ON  SHIP  PROCESSES 


The  on  ship  process  reflects  all  of  the  processes  in  the  AEGIS  software 
maintenance  and  update  process  that  would  take  place  aboard  a  U.S.  Naval  vessel.  The 
detection  of  equipment  and  software  failures  and  the  effect  of  these  failures  on  the 
mission  capability  of  the  AEGIS  system  are  made  by  two  departments.  The  anomaly 
identification  and  repair  function  works  with  the  detection,  reporting  and  resolution  of 
software  anomalies  function.  Casualty  Report  (CASREP)  procedures,  under  normal  ship 
operations,  are  explained  below. 

1.  Software  Anomaly  Detected 

Software  anomalies  can  be  detected  through  many  methods.  In  most  cases  the 
software  and  test  engineers  in  the  test  phase  of  software  maintenance  and  update  process 
detect  these  anomalies.  Software  anomalies  can  also  be  detected  during  the  combat 
systems  integration  testing  after  shipboard  delivery.  This  is  the  first  time  that  the 
software  is  in  its  shipboard  configuration  and  allows  for  fully  fielded  software  updates  to 
be  operationally  tested.  On  rare  occasions  software  anomalies  can  also  be  detected 
during  nonnal  shipboard  operation  by  Surface  Warfare  fleet  operators. 

2.  Cause  of  Anomaly  Determined 

Personnel  who  have  observed  the  anomaly  when  the  software  is  in  its  shipboard 
configuration  attempt  to  collect  data  regarding  the  anomaly  and  also  attempt  to  trace  the 
anomaly  to  its  source. 

3.  Software  Bug  Report  Submitted 

All  data  and  information  surrounding  the  anomaly  and  its  source  are  collected. 
The  configuration  and  status  of  the  AEGIS  system  as  well  as  any  environmental  data  is 
also  gathered  and  reported  to  the  Program  Manager. 


31 


F. 


OFF  SHIP  AGGREGATE  PROCESSES 


The  following  represents  an  aggregate  software  maintenance  and  update  process 
procedure  taken  at  the  NSWC,  Dahlgren,  V.A.  to  process  software  anomalies  when  an 
anomaly  has  been  detected  on  software  in  its  shipboard  configuration.  The  aggregate 
process  executed  at  NSWC,  Dahlgren,  V.A.  will  be  further  decomposed  and  explained  in 
the  next  section. 

1.  Software  Anomaly  Verified 

The  Project  Manager  and  the  project  team  work  to  recreate  the  conditions  in  a  lab 
that  were  documented  in  the  Software  Bug  Report  in  an  effort  to  recreate  the  software 
anomaly.  If  the  software  anomaly  is  verified  the  software  maintenance  and  update 
process  will  continue. 

2.  Anomaly  Appended  to  Working  List  of  Known  Issues 

The  software  anomaly  is  documented  in  the  Computer  Program  Change  Request 
(CPCR).  Software  anomalies  are  tracked  in  the  ACCESS/STARSY  database.  The  CPCR 
is  also  assessed  by  the  Joint  Change  Review  Board  (JCRB)  and  the  Software 
Configuration  Change  Board  (SCCB)  for  inclusion  in  the  baseline. 

3.  Workaround  Developed 

If  CPCR  is  not  corrected,  then  if  a  workaround  exists  to  allow  for  avoidance  of 
the  anomaly,  it  is  documented  in  the  database  and  included  in  the  Computer  Program 
Design  Document  (CPDD)/Crew  Brief. 

4.  New  Software  Version  Developed 

Teams  of  software  programmers  develop  new  versions  of  the  software  which  not 
only  serve  to  fix  anomalies,  but  also  implement  new  functionality  and  exploit  new 
technologies. 
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5. 


Directed  Software  Anomalies  are  Resolved 


Anomalies  that  are  evident  in  the  new  software  and  are  found  through 
certification  testing  in  labs  are  resolved  and  the  required  changes  are  made  to  the  new 
software. 


6.  New  Software  Fielded  to  Units 

Teams  of  contractors  and  support  personnel  arrive  aboard  ship  to  deliver  new 
software,  install  the  new  software  and  conduct  crew  briefs  and  training. 

G.  OFF  SHIP  PROCESS  DECOMPOSITION 

After  the  initial  process  model  depicted  above  in  Table  7  was  developed,  it 
became  apparent  after  numerous  conversations  with  subject  matter  experts  that  the  most 
significant  improvements  to  the  software  maintenance  and  update  process  would  result  in 
restructuring  the  Off  Ship  component.  Due  to  this,  a  higher  level  of  decomposition  was 
needed  on  for  the  Off  Ship  processes  occurring  at  NSWC,  Dahlgren  V.A.  Subject  matter 
experts  were  again  consulted  and  a  more  detailed  process  model  for  the  AEGIS  software 
maintenance  and  update  process  that  occurred  at  NSWC,  Dahlgren  V.A.  was  developed. 
The  more  detailed  Off  Ship  AEGIS  software  maintenance  and  update  process  is  depicted 
in  Table  8  and  explained  below. 
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►  Shipboard  Delivery 


Combat  System 
Integration  Test 


Color  Code 

Requirements  Definition  Phase  -  Green 
Design  Phase  -  Red 
Test  Phase  -  Blue 

Implementation  and  Installation  Phase  -  Yellow 


Table  8.  AEGIS  software  maintenance  and  update  process  (Off  Ship) 
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1.  Inputs  and  Requirements  Gathered 


Fleet  inputs,  external  interface  requirements  and  new  system  requirements  are  all 
gathered  during  this  process. 

2.  System  Design  Review 

This  is  a  scheduled  review  that  is  technical  in  nature.  “This  review  shall  be 
conducted  to  evaluate  the  optimization,  correlation,  completeness,  and  risks  associated 
with  the  allocated  technical  requirements.  Also  included  is  a  summary  review  of  the 
system  engineering  process  which  produced  the  allocated  technical  requirements  of  the 
engineering  planning  for  the  next  phase  of  effort.”30 

3.  ECPs,  SCPs,  ICRs 

This  process  documents  any  changes  that  need  to  be  made  to  the  software  after  it 
has  undergone  the  system  design  review.  These  will  be  documented  through  an 
Engineering  Change  Proposal  (ECP),  Software  Change  Proposal  (SCP)  and  an  Interface 
Change  Request  (ICR). 

4.  Approval  Process 

The  change  proposals  and  requests  are  then  sent  through  an  approval  process 
where  the  SCP  awaits  Software  Configuration  Change  Board  (SCCB)  approval,  and  the 
ICR  undergoes  Integration  Working  Group  (IWG)  approval.  The  aggregate  approvals  are 
then  sent  to  PMS  442  for  approval.  PMS  442  is  the  acquisition  organization  responsible 
for  that  product,  in  this  case  it  would  be  NAVSEA  program  management. 


30  Department  of  Defense  1985. 
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5.  Design  Review 


This  is  the  first  step  in  the  design  phase  of  the  AEGIS  software  maintenance  and 
update  process.  The  design  review  process  includes  a  preliminary  design  review  (PDR), 
and  a  Critical  Design  Review  (CDR). 

6.  Design  Walkthrough 

The  design  walkthrough  process  includes  writing  and  inspecting  code,  unit  test 
and  analysis  and  the  debugging  of  code. 

7.  PDS/SDD 

The  previous  design  walkthrough  process  produces  the  Preliminary  Design 
Specification  (PDS)  and  the  Software  Design  Document  (SDD) 

8.  Develop  Test  Plan 

This  is  the  first  process  in  the  test  phase  of  the  software  maintenance  and  update 
process.  In  this  process  a  test  plan  is  developed  using  test  specifications  and  the  test  case 
design  process.  For  the  purposes  of  this  thesis,  it  is  not  necessary  to  describe  the  test  case 
design  process. 


9.  Test  Procedures 

The  test  procedures  are  the  output  of  the  develop  test  plan  process.  These 
procedures  will  later  be  utilized  in  the  test  execution  and  data  analysis  process. 

10.  Test  Readiness  Review  Process 

The  test  plan  and  test  procedures  are  reviewed  in  order  to  make  sure  that  the  test 
process  will  be  most  effective.  In  order  to  achieve  maximum  efficiency,  collaborative 
testing  and  data  analysis  are  included. 
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11.  Identify/Resolve  Issues 


This  process  includes  an  assessment  of  any  CPCRs  for  possible  program  update 
and  also  the  certification  impact  of  any  CPCRs. 

12.  Certification  Impact  Decided 

If  any  of  the  CPCRs  are  determined  to  have  the  potential  for  a  high  certification 
impact  then  the  program  must  be  updated  before  it  can  be  sent  to  the  test  execution  and 
data  analysis  portion  of  the  software  maintenance  and  update  process.  If  the  CPCRs  are 
not  determined  to  have  a  high  certification  impact  the  software,  then  is  sent  directly  to 
test  execution  and  data  analysis. 

13.  Update  Program  (High  Certification  Impact) 

For  software  that  contains  CPCRs  that  are  determined  to  have  a  high  certification 
impact,  the  software  must  be  updated.  This  includes  another  unit  test  and  analysis  and  an 
assessment  of  the  certification  retest. 

14.  Updated  Program 

Once  the  program  is  updated  and  the  unit  test  and  analysis  and  an  assessment  of 
the  certification  retest  has  been  conducted,  the  software  is  then  sent  to  test  execution  and 
data  analysis. 


15.  Test  Execution  and  Data  Analysis 

The  software  is  tested  in  lab  conditions  in  order  to  detect  any  potential  problems 
that  might  arise  before  the  software  is  fully  fielded.  It  consists  of  three  sub  processes. 
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a.  Software  Anomaly  Discovered 

A  software  anomaly  is  found  under  lab  conditions. 

b.  Anomaly  Documented  in  CPCR  Database 

A  CPCR  is  generated  for  the  anomaly  and  is  then  entered  in  the 
ACCESS/STARSY  database. 

c.  CPCR  Assessed 

The  CPCR  is  prioritized  and  its  certification  impact  is  assessed.  The 
CPCRs  operational  impact  is  also  assessed  and,  if  possible,  a  workaround  is  established. 

16.  Document  Results 

The  results  from  the  test  execution  and  data  analysis  portion  of  the  software 
maintenance  and  update  process  are  documented. 

17.  Conduct  Functional  Area  Assessment 

This  process  is  an  analysis  conducted  at  a  higher  level  in  order  to  prepare  the 
software  for  the  certification  panel  review. 

18.  Conduct  Certification  Panel 

A  certification  panel  assesses  the  software’s  results  from  the  test  execution  and  data 
analysis  process  and  certifies  the  software  for  fielding. 

19.  PAT/FQT 

A  Preliminary  Acceptance  Test  (PAT)  and  Functional  Quality  Testing  (FQT)  is 
then  conducted  on  the  software. 
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20.  Data  Analysis 


The  data  collected  from  the  PAT  and  the  FQT  is  assessed  and  analyzed  in 
preparation  for  the  Lab  Combat  Systems  Integration  Test. 

21.  Lab  Combat  System  Integration  Test 

This  process  includes  any  final  testing  that  occurs  in  the  lab  environment 
including  any  software  trouble  reports  (STR)  that  are  collected.  CPSA  Analysis  and  a 
CPSA  report. 

22.  Shipboard  Delivery 

The  new  software  is  fielded  to  operational  units  and  installed  by  teams  of 
contractors  and  support  personnel.  Crew  briefs  and  training  are  also  conducted 

23.  Shipboard  Combat  System  Integration  Test 

The  software  is  tested  in  its  fully  fielded  shipboard  configuration.  Due  to  the  fact 
that  this  is  the  first  time  the  software  is  in  the  shipboard  configuration  it  must  be  tested 
for  functionality.  This  also  ensures  that  it  is  interoperable  with  all  combat  systems  already 
in  place  on  the  ship. 

H.  “AS  IS”  KVA  ANALYSIS  -  ON  SHIP  AND  OFF  SHIP  AGGREGATE 

PROCESS  MODEL 

An  analysis  of  each  sub  process  in  the  AEGIS  software  maintenance  and  upgrade 
process  for  the  On  Ship  and  Off  Ship  aggregate  process  model  is  provided  in  the 
following  tables.  The  information  provided  for  each  analysis  was  produced  through 
discussions  with  subject  matter  experts.  Each  category  for  the  KVA  analysis  is  defined 
below. 
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1. 


Title  of  Head  Process  Executer 


The  “Title  of  Head  Process  Executer”  category  represents  the  job  title  of  the 
person  executing  or  overseeing  the  execution  of  the  specific  process  or  sub  process.  The 
process  executors  pay  grade  is  indicated  next  to  their  job  title.  For  purposes  of  this  thesis, 
we  used  pay  grades  that  erred  on  the  high  side  to  be  conservative.  If  several  executors 
with  different  pay  grades  were  executing  the  same  process  then  the  highest  pay  grade  was 
used  as  a  baseline  for  that  process  executor.  This  produces  the  most  conservative  KVA 
results.  Some  other  basic  assumptions  for  this  category  were: 

*  Each  pay  grade  was  to  be  at  a  Step  6  level  within  their  respective  pay 
grade 

*  Base  pay  was  location  adjusted  for  San  Diego,  CA  as  a  baseline 

*  Civilian  software  contractor  salary  market  comparables  were  estimated  to 
be  175%  of  government  salary 

2.  Number  of  Employees 

The  “Number  of  Employees”  category  represents  the  number  of  government 
employees  or  contractors  which  are  involved  in  the  specific  sub  process.  If  more  than 
one  person  was  involved  in  both  the  parent  and  specific  sub  processes,  that  person  is 
documented  separately  for  each  sub  process. 

3.  Rank  Order  of  Difficulty 

An  ordinal  ranking  of  the  relative  difficulty  of  learning  each  of  the  processes  is 
collected  and  used  to  ensure  that  the  “Relative  Learning  Time”  and  “Actual  Average 
Training  Period”  estimates  are  reliable.  By  allowing  the  subject  matter  experts  to  rank 
each  of  the  sub  processes  ( 1  being  the  least  complex)  outside  of  the  context  of  time  units 
a  correlation  can  be  made  between  the  “Rank  Order  of  Difficulty”  and  the  “Relative 
Learning  Time.”  If  a  correlation  of  80%  is  achieved  the  results  appear  to  be  reliable  and 
the  “Relative  Learning  Time”  can  be  considered  an  accurate  description  of  the  relative 
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difficulty  of  the  sub  processes.  If  a  correlation  of  80%  is  not  achieved,  the  results  must 
be  closely  scrutinized  and  the  subject  matter  experts  must  be  resurveyed  and  possibly 
given  a  more  in-depth  explanation  of  the  concept  of  “Relative  Learning  Time.” 

4.  Relative  Learning  Time 

The  “Relative  Learning  Time”  category  represents  a  distributed  relative  amount 
of  100  hours  of  learning  time  among  the  processes.  “Relative  Learning  Time”  assumes 
an  “average  person”  will  leam  all  he/she  needs  to  know  to  successfully  complete  all  the 
tasks  in  each  process.  This  learning  time  estimate  includes  the  time  it  would  take  to  learn 
how  to  produce  the  same  output  that  any  automation  (e.g.  infonnation  systems)  currently 
produces.  The  100-hour  learning  period  is  distributed  according  to  how  difficult  and 
complex  the  processes  are  for  the  “average  person”  to  learn.  The  purpose  is  to  determine 
“Relative  Learning  Times”  for  each  process  given  the  100-hour  total.  This  helps  identify 
the  most  complex  processes  and  can  be  used  as  another  internal  reliability  measure. 

5.  Actual  Average  Training  Period 

The  “Actual  Average  Learning  Time”  is  what  the  actual  average  training  time  in 
hours  is  for  the  “average  person”  for  each  process.  This  would  be  for  a  new  employee 
with  no  background  who  would  be  required  to  learn  everything  to  produce  the  outputs  of 
the  given  processes.  Learning  time  includes  both  formal  training  and  on  the  job  training. 

The  results  from  “Relative  Learning  Time”  and  “Actual  Average  Learning  Time” 
are  also  correlated.  If  a  correlation  of  80%  is  achieved  the  results  appear  to  be  reliable 
and  the  “Actual  Average  Learning  Time”  can  be  considered  an  accurate  description  of 
the  “Relative  Learning  Time”  of  the  sub  processes.  If  a  correlation  of  80%  is  not 
achieved,  the  results  must  be  closely  scrutinized  and  the  subject  matter  experts  must  be 
resurveyed  and  possibly  given  a  more  in  depth  explanation  of  the  concept  of  “Relative 
Learning  Time”  along  with  “Actual  Average  Learning  Time.”  In  some  cases,  subject 
matter  experts  may  associate  “Actual  Average  Learning  Time”  with  a  school  or  training 
period  associated  with  the  process.  These  schools  and  training  periods  are  generally 
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conducted  over  a  uniform  length  of  time  and  do  not  accurately  reflect  the  “Actual 
Average  Learning  Time.”  Assumptions  include: 

*  On  the  job  training  is  estimated  to  be  50%  of  the  time  learning  the  task 
and  50%  of  the  time  actually  performing  the  task. 

*  On  the  job  training  was  conducted  8  hours  a  day,  5  days  a  week,  and  50 
weeks  per  year. 

*  On  the  job  training  occurs  over  a  one  year  period. 

6.  Percentage  Automation 

Each  sub  process  has  a  “Percentage  Automation”  associated  with  it  between  0  and 
100.  This  number  captures  the  knowledge  that  is  embedded  in  any  information 
technology  so  that  it  can  be  accounted  for  in  later  calculations.  This  number  represents 
the  percentage  of  information  technology  that  it  utilized  so  that  a  process  executor  would 
not  have  to  accomplish  the  task.  For  example,  a  process  that  has  100%  automation  would 
not  require  any  process  executors  and  would  be  accomplished  fully  by  the  automation 
tools  listed  for  that  process.  If  a  process  has  0%  automation,  no  automation  tools  are 
utilized  and  the  process  is  totally  executed  by  the  process  executors.  These  numbers  are 
estimates  based  on  subject  matter  expert’s  observations  and  experience.  One  basic 
assumption  associated  with  this: 

*  “Replacement  Technology,”  is  automation  that  will  reduce  the  number  of 
process  executors  associated  with  the  process  without  increasing  the  output  of  the 
process. 


7.  Times  Performed  in  a  Year 

The  “Times  Performed  in  a  Year”  category  represents  the  number  of  times  each 
sub  process  is  acted  upon  by  a  head  process  executor  in  a  given  year.  The  values  were 
obtained  by  asking  subject  matter  experts  for  their  inputs  to  determine  a  valid  estimate  for 
the  year  long  period. 
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8.  Average  Time  to  Complete 


Each  time  a  sub  process  is  acted  upon  (as  indicated  in  the  “Times  Perfonned  in  a 
Year”  category)  there  is  a  specific  amount  of  time  that  it  takes  for  each  sub  process  to  be 
satisfactorily  completed.  This  category  represents  the  number  of  hours  it  takes  a  person 
trained  in  each  process/sub  process  to  complete  each  task. 

9.  Automation  Tools 

The  “Automation  Tools”  category  represents  any  tools  such  as  MS  Office,  Visio, 
SIPR  Database  or  Belvoir  Paperless  Office.  This  is  used  as  a  baseline  for  any  automation 
tools  that  are  already  in  use  for  the  process  and  may  provide  insight  for  the 
implementation  of  other  automation  tools. 

10.  Total  Learning  Time  (TLT) 

This  category  is  produced  by  dividing  the  “Actual  Average  Learning  Time”  by 
the  “Percent  Automation.”  Because  we  assume  “replacement  technology,”  the  formula 
used  to  determine  TLT  is  “Actual  Average  Learning  Time”/(1 -“Percent  Automation”). 
This  provides  a  total  time,  in  hours,  for  each  process  to  include  the  learning  time  that  is 
present  in  the  automation  tools.  For  example,  if  it  takes  one  hour  to  learn  a  system  that  is 
50%  automated  then  the  total  learning  time  associated  with  the  process  is  two  hours,  one 
hour  associated  with  the  process  executors  and  one  hour  associated  with  the  automation 
tools. 


11.  Total  Knowledge 

This  category  is  a  representation,  in  hours,  of  all  of  the  knowledge  for  each 
process  that  occurs  over  the  one  year  time  frame  in  which  the  survey  encompasses.  The 
“Number  of  Employees”  category  is  multiplied  by  the  “Times  Perfonned  in  a  Year,”  and 
the  “Total  Learning  Time”  categories. 
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12.  Personnel  Cost 


This  number  represents  the  costs  that  are  associated  with  the  government 
employees  associated  with  each  process.  This  number  is  calculated  by  multiplying  the 
employee’s  hourly  wage  with  the  “Average  Time  to  Complete,”  the  “Number  of 
Employees,”  and  the  “Times  Preformed  in  a  Year”  categories.  This  number  shows  the 
cost  of  process  only  associated  with  personnel  over  the  course  of  a  year.  Even  though 
personnel  may  be  involved  with  other  tasks  during  their  employment,  this  number  shows 
the  wages  paid  only  based  on  their  work  with  the  specified  process.  Assumptions 
include: 

*  An  average  employee  works  250  days  per  year 

*  Wages  are  adjusted  for  the  location  of  San  Diego,  C.A. 

13.  Other  Costs 

The  “Other  Costs”  category  represents  the  cost  of  the  process  executors  utilizing 
workstations  that  can  access  the  Navy  Marine  Corps  Intranet  (NMCI).  This  category  was 
calculated  by  taking  the  average  cost  per  seat  associated  with  NMCI  and  multiplying  it  by 
the  “Number  of  Employees”  category.  Assumptions  include: 

*  NMCI  cost  per  seat  is  approximately  $300031 

*  Assumes  a  “Red  Seat”  -  Pentium  800MHz.  This  provides 
performance  for  use  with  2-D  and  light  3-D  graphics  or 
engineering  related  applications,  applications  that  require 
additional  processing  capability.  32 


3 '  Navy  Marine  Corps  Intranet 
32  Navy  Marine  Corps  Intranet 
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14.  Total  Costs 


The  “Total  Costs”  category  represents  the  total  costs  of  the  process.  This 
category  was  calculated  by  adding  the  “Personnel  Costs”  category  with  the  “Other  Costs” 
category. 


15.  Price 

This  category  represents  the  price  that  it  would  cost  if  the  process  was  executed 
by  civilian  contractors  rather  than  government  employees.  The  “Price”  category  was 
calculated  much  in  the  same  way  that  the  “Personnel  Cost”  category  was  calculated, 
except  using  contractor  hourly  wages  in  lieu  of  government  employee  hourly  wages. 
This  category  was  calculated  by  multiplying  the  contractor  wage  per  hour  by  the 
“Number  of  Employees,”  the  “Average  Time  to  Complete,”  and  the  “Times  Performed  in 
a  Year”  categories.  Assumptions  include: 

*  Civilian  software  contractor  market  comparables  were  estimated  to  be 
175%  of  government  salary 

16.  Denominator 

This  category  shows  the  cost  associated  with  producing  the  output  of  each 
process.  It  is  the  same  as  the  “Total  Costs”  category. 

17.  Numerator 

The  “Numerator”  category  is  the  “percentage  of  the  revenue  or  sales  dollar 
allocated  to  the  amount  of  knowledge  required  to  obtain  the  outputs  of  a  given  process  in 
proportion  to  the  total  amount  of  knowledge  required  to  generate  the  corporation’s 
salable  outputs.”33  For  the  purposes  of  this  thesis,  the  revenue  allocated  to  the  amount  of 
knowledge  can  be  compared  to  the  amount  of  knowledge  that  is  present  in  each  process 
or  sub  process.  This  can  also  be  thought  of  as  the  total  knowledge  multiplied  by  the  price 


33  Housel,  Thomas  2001;  45. 
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of  each  common  unit.  This  value  was  calculated  by  first  finding  the  price  of  each 
common  unit.  The  price  per  common  unit  was  calculated  by  dividing  the  total 
knowledge  into  the  total  price.  The  formula  is  (price  of  the  entire  process)/(total 
knowledge  of  the  entire  process).  The  “Numerator”  was  then  calculated  by  multiplying 
the  total  knowledge  associated  with  each  sub  process  with  the  price  of  each  common  unit. 
Establishing  a  price  per  common  unit  is  important  when  developing  the  to-be  model 
which  will  be  discussed  in  the  Chapter  IV. 

I.  ROK 

With  each  process  or  sub  process  there  is  both  a  cost  and  a  revenue  associated 
with  producing  an  output.  The  Return  on  Knowledge  (ROK)  provides  a  representation  of 
how  well  the  assets  within  a  process  are  distributed  in  relation  to  one  another  by  utilizing 
the  costs  and  revenues  associated  with  each  sub  process.  The  ROK  is  calculated  by 
dividing  the  “Numerator”  by  the  “Denominator.”  ROK’s  can  be  compared  within  a 
process  to  determine  which  processes  are  utilizing  assets  in  an  efficient  manner  and 
which  processes  need  to  be  changed,  perhaps  by  the  utilization  of  automation  tools,  in 
order  to  improve  efficiency.  Although  ROK  is  a  very  valuable  tool,  a  low  ROK  does  not 
dictate  that  a  process  is  in  need  of  increased  automation,  but  serves  as  an  indicator  that 
the  process  should  be  analyzed  more  closely  to  discover  if  process  efficiency  can  be 
improved. 

J.  ROI 

“ROI”  or  return  on  investment  is  a  common  accounting  term  that  is  widely 
understood  by  the  financial  community.  For  this  reason,  it  is  a  slightly  more  meaningful 
number  than  “ROK.”  Essentially,  it  is  a  very  similar  number  to  “ROK,”  just  a  different 
unit  of  measure.  In  financial  tenns,  “ROI”  is  the  profit  or  loss  resulting  from  an 
investment  transaction,  usually  expressed  as  an  annual  percentage  return.  “ROI”  is  a 
return  ratio  that  compares  the  net  benefits  of  a  project  verses  its  total  costs.  In  financial 
terms,  “ROI”  is  calculated  by  profit  minus  investment  all  divided  by  investment.  For  the 
purposes  of  KVA,  “ROI”  is  calculated  by  the  “Numerator”  minus  the  “Denominator”  all 
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dived  by  “Denominator.”  Much  like  “ROK’s,”  “ROI’s”  can  be  compared  within  a 
process  to  determine  which  processes  are  utilizing  assets  in  an  efficient  manner  and 
which  processes  need  to  be  changed,  perhaps  by  the  utilization  of  automation  tools,  in 
order  to  improve  efficiency. 

K.  “AS  IS”  PROCESS  DATA  -  ON  SHIP  AND  OFF  SHIP  AGGREGATE 

PROCESS  MODEL 

Each  of  the  AEGIS  software  maintenance  and  upgrade  processes  and  sub 
processes  will  be  presented  for  evaluation.  The  ROKs  and  ROIs  that  were  calculated  for 
each  process  and  sub  process  will  be  utilized  to  find  the  differences  in  efficiency  for  each 
of  the  processes  and  sub  processes.  The  analysis  of  the  “As  Is”  process  data  will  provide 
insight  on  the  amount  and  location  of  knowledge  assets  throughout  the  AEGIS  software 
maintenance  and  upgrade  process. 

1.  On  and  Off  Ship  Aggregate  Process 

Table  9  and  10  depict  the  on  and  off  ship  aggregate  processes  included  in  the 
KVA  analysis  for  the  AEGIS  software  maintenance  and  upgrade  process  that  occurs  for 
one  AEGIS  ship. 
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Process 

Title  of  Head  Process  Executer 

Number  of  Employees 

Rank  Order  of  Difficulty 

Relative  Learning 
Time 

Actual  Average 
Learning  Time 

Percentage 

Automation 

Times  Performed  In  a 

Year 

Average  Time  to 
Complete 

Automation  Tools 

Software  Anomaly  Detected 

Project  Manager  (GS-11) 

2 

4 

10 

300 

0% 

10 

20 

N/A 

Cause  of  Anomaly  Determined 

Technology  Director  (GS-12/13) 

5 

8 

20 

500 

30% 

25 

40 

Advanced  Software 

Software  Bug  Report  Submitted 

Project  Manager  (GS-11) 

1 

3 

10 

300 

10% 

10 

4 

MS  Word 

Anomaly  Verified 

Project  Manager  (GS-11 /1 2) 

2 

5 

10 

300 

0% 

10 

20 

N/A 

Anomaly  Appended  to  Working  List  of  Known  Issues 

Fleet  Support  (GS-9/10) 

1 

2 

5 

160 

10% 

10 

4 

Excel 

Workaround  Developed 

Project  Manager  (GS-11/12) 

2 

6 

10 

160 

0% 

10 

20 

N/A 

New  Software  Version  Developed 

Lead  Programmer  (GS-13/14) 

10 

9 

20 

640 

20% 

2 

200 

Compiler 

Known  Anomalies  are  Resolved 

Lead  Programmer  (GS-12/13) 

5 

7 

10 

500 

20% 

2 

200 

N/A 

New  Software  Version  Fielded  to  Units 

Fleet  Support  (GS-9/10) 

3 

1 

5 

160 

10% 

20 

80 

Tracking 

Correlation  (Ordinal  to  RLT)  0.877032523 

Correlation  (RLT  to  MTP)  0.845793835 


Process  (continued) 

TLT 

Total  Knowledge 

Personel  Cost 

Other  Costs 

Total  Costs 

Price 

Denominator 

Numerator 

ROK 

ROI 

Software  Anomaly  Detected 

300 

6000 

$  13,701.00 

$  600.00 

$  14,301.00 

$  23,976.75 

14301 

56826.27304 

397% 

297% 

Cause  of  Anomaly  Determined 

714 

89286 

$  244,096.88 

$  7,500.00 

$  251,596.88 

$  427,169.53 

251596.875 

845629.0632 

336% 

236% 

Software  Bug  Report  Submitted 

333 

3333 

$  1,370.10 

$  60.00 

$  1,430.10 

$  2,397.68 

1430.1 

31570.15169 

2208% 

2108% 

Anomaly  Verified 

300 

6000 

$  16,421.50 

$  600.00 

$  17,021.50 

$  28,737.63 

17021.5 

56826.27304 

334% 

234% 

Anomaly  Appended  to  Working  List  of  Known  Issues 

178 

1778 

$  1,247.00 

$  60.00 

$  1,307.00 

$  2,182.25 

1307 

16837.41424 

1288% 

1188% 

Workaround  Developed 

160 

3200 

$  16,421.50 

$  600.00 

$  17,021.50 

$  28,737.63 

17021.5 

30307.34562 

178% 

78% 

New  Software  Version  Developed 

800 

16000 

$  230,750.00 

$  6,000.00 

$  236,750.00 

$  403,812.50 

236750 

151536.7281 

64% 

-36% 

Known  Anomalies  are  Resolved 

625 

6250 

$  97,638.75 

$  3,000.00 

$  100,638.75 

$  170,867.81 

100638.75 

59194.03442 

59% 

-41% 

New  Software  Version  Fielded  to  Units 

178 

10667 

$  149,640.00 

$  7,200.00 

$  156,840.00 

$  261,870.00 

156840 

101024.4854 

64% 

-36% 

totals 

142513 

1  771,286.73 

T  25,620.00 

1  796,906.73 

1  1,349,751.77 

796906.725 

1349751.769 

169% 

5K 

Personel  Costs  Base  Pay 


GS9 

$ 

45,294.00  $ 

GS 10 

$ 

49,880.00  $ 

GS 11 

$ 

54,804.00  $ 

GS  12 

$ 

65,686.00  $ 

GS  13 

$ 

78,111.00  $ 

GS  14 

$ 

92,300.00  $ 

Price  Per  Common  Unit  $ 

Table  9. 


Location  Adjusted  Hourly  Wage  Contractor  Wage 


56,617.50  $ 

28.31  $ 

49.54 

62,350.00  $ 

31.18  $ 

54.56 

68,505.00  $ 

34.25  $ 

59.94 

82,107.50  $ 

41.05  $ 

71.84 

97,638.75  $ 

48.82  $ 

85.43 

115,375.00  $ 

57.69  $ 

100.95 

9.47 

Off  Ship  Aggregate  Process  (One  ship) 
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2. 


On  and  Off  Ship  Aggregate  Process  for  Entire  AEGIS  Fleet 


Table  10  depicts  the  further  decomposed  off  ship  process  included  in  the  KVA 
analysis  for  the  AEGIS  software  maintenance  and  upgrade  process  that  is  scaled  to 
include  all  AEGIS  ships  in  the  U.S.  Fleet.  Assumptions  for  scaling  the  process  data 
include: 

*  Updates  are  delivered  to  each  ship  twice  a  year. 

*  Each  update  contains  five  anomaly  fixes. 

*  The  “New  Software  Version  Fielded  to  Units”  occurs  168  times  per  year 
based  on  84  AEGIS  ships  and  two  updates  per  year. 
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Process 

Title  of  Head  Process  Executer 

Number  of  Employees 

Rank  Order  of  Difficulty 

Relative  Learning 
Time 

Actual  Average 
Learning  Time 

Percentage 

Automation 

Times  Performed  In  a 
Year 

Average  Time  to 
Complete 

Automation  Tools 

Software  Anomaly  Detected 

Project  Manager  (GS-11) 

2 

4 

10 

300 

0% 

10 

20 

N/A 

Cause  of  Anomaly  Determined 

Technology  Director  (GS-12/13) 

5 

8 

20 

500 

30% 

25 

40 

Advanced  Software 

Software  Bug  Report  Submitted 

Project  Manager  (GS-11) 

1 

3 

10 

300 

10% 

10 

4 

MS  Word 

Anomaly  Verified 

Project  Manager  (GS-11/12) 

2 

5 

10 

300 

0% 

10 

20 

N/A 

Anomaly  Appended  to  Working  List  of  Known  Issues 

Fleet  Support  (GS-9/10) 

1 

2 

5 

160 

10% 

10 

4 

Excel 

Workaround  Developed 

Project  Manager  (GS-11/12) 

2 

6 

10 

160 

0% 

10 

20 

N/A 

New  Software  Version  Developed 

Lead  Programmer  (GS-13/14) 

10 

9 

20 

640 

20% 

2 

200 

Compiler 

Known  Anomalies  are  Resolved 

Lead  Programmer  (GS-12/13) 

5 

7 

10 

500 

20% 

2 

200 

N/A 

New  Software  Version  Fielded  to  Units 

Fleet  Support  (GS-9/10) 

3 

1 

5 

160 

10% 

20 

80 

Tracking 

Correlation  (Ordinal  to  RLT)  0.877032523 

Correlation  (RLT  to  AATP)  0.845793835 


Process  (continued) 

TLT 

Total  Knowledge 

Personel  Cost 

Other  Costs 

Total  Costs 

Price 

Denominator 

Numerator 

ROK 

ROI 

Software  Anomaly  Detected 

300 

6000 

S  13,701.00 

$  600.00 

3 

14,301.00 

$  23,976.75 

14301 

4773406.936 

33378% 

33278% 

Cause  of  Anomaly  Determined 

714 

89286 

S  244,096.88 

$  7,500.00 

3 

251,596.88 

$  427,169.53 

251596.875 

71032841.31 

28233% 

28133% 

Software  Bug  Report  Submitted 

333 

3333 

$  1,370.10 

$  60.00 

3 

1,430.10 

$  2,397.68 

1430.1 

2651892.742 

185434% 

185334% 

Anomaly  Verified 

300 

6000 

$  16,421.50 

$  600.00 

3 

17,021.50 

$  28,737.63 

17021.5 

4773406.936 

28043% 

27943% 

Anomaly  Appended  to  Working  List  of  Known  Issues 

178 

1778 

3  1,247.00 

$  60.00 

3 

1,307.00 

$  2,182.25 

1307 

1414342.796 

108213% 

108113% 

Workaround  Developed 

160 

3200 

3  16,421.50 

$  600.00 

3 

17,021.50 

$  28,737.63 

17021.5 

2545817.032 

14956% 

14856% 

New  Software  Version  Developed 

800 

16000 

3  230,750.00 

$  6,000.00 

3 

236,750.00 

$  403,812.50 

236750 

12729085.16 

5377% 

5277% 

Known  Anomalies  are  Resolved 

625 

6250 

3  97,638.75 

$  3,000.00 

3 

100,638.75 

$  170,867.81 

100638.75 

4972298.891 

4941% 

4841% 

New  Software  Version  Fielded  to  Units 

178 

10667 

5  149,640.00 

$  7,200.00 

3 

26,349,120.00 

$  261,870.00 

26349120 

8486056.775 

32% 

-68% 

Totals 

'  142515 

1  771,286.73 

T  25,620.00 

J 

26,989,186.73 

1  1,349,751.77 

26989186.73 

113379148.6 

WI» 

as 

Personel  Costs 

Base  Pay 

GS9 

$ 

45,294.00 

GS10 

$ 

49,880.00 

GS11 

$ 

54,804.00 

GS12 

$ 

65,686.00 

GS13 

$ 

78,111.00 

GS14 

$ 

92,300.00 

Price  Per  Common  Unit 

Table  10. 


Location  Adjusted  Hourly  Wage  Contractor  Wage 


56,617.50  $ 

28.31  $ 

49.54 

62,350.00  J 

31.18  $ 

54.56 

68,505.00  $ 

34.25  $ 

59.94 

82,107.50  $ 

41.05  $ 

71.84 

97,638.75  $ 

48.82  $ 

85.43 

115,375.00  $ 

57.69  $ 

100.95 

9.47 

Off  Ship  Aggregate  Process  (All  Ships) 
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3. 


Off  Ship  Process  Decomposition 


Table  1 1  depicts  the  further  decomposed  off  ship  process  included  in  the  KVA 
analysis  for  the  AEGIS  software  maintenance  and  upgrade  process  that  occurs  for  one 
AEGIS  ship. 
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Process 

Title  of  Head  Process  Executer 

Number  of  Employees 

Rank  Order  of  Difficulty 

Relative  Learning 
Time 

Actual  Average 
Learning  Time 

Percentage 

Automation 

Times  Performed  In  a 
Year 

Average  Time  to 
Complete 

Automation  Tools 

Cost  of  IT 

Fleet  Inputs/External  Interface  Requirements/New  System  Resources 

Program  Manager  (ND-IV) 

15 

2 

1 

500 

15% 

3 

335 

Databases  to  ID  issues 
for  inclusion/fleet 
issues/etc 

10000 

System  Design  Review 

Systems  Engineer  (ND-IV) 

50 

4 

2 

500 

5% 

1 

335 

database/Adjudication 

10000 

ECP's/SCP’s/ICR's 

Systems  Engineer  (ND-IV) 

25 

250 

25 

60 

ACS  IS 

5U000 

Approval  Process  (Including  FPRG,  SCCB,  IWG,  PMS  422) 

PM,  PMS  422,  LSEA  (ND-V) 

50 

8 

8 

835 

15% 

50 

8 

ACSIS 

45000 

Design  Review  (Including  PDR,  DDWS,  CDR) 

PM  (ND-IV) 

50 

5 

5 

500 

5% 

3 

120 

Comment 

database/Adjudication 

tracker 

10000 

Design  Walkthrough  (Including  Writing  and  Inspecting  Code,  Unit  Test  and  Analysis,  Debugging  Code) 

Devloper  (ND-IV) 

8 

7 

7 

1000 

15% 

50 

80 

MS  Word 

0 

Develop  Test  Plan  (Including  Test  Specifications  and  Test  Case  Design) 

Test  Engineer  (ND-IV) 

2 

12 

10 

1000 

5% 

50 

1000 

MS  Word 

0 

Test  Procedures 

1  est  Engineer  (ND-IV) 

2 

13 

10 

1000 

5% 

50 

1000 

MS  Word 

0 

Test  Readiness  Review 

PM  (ND-IV) 

30 

3 

1 

500 

5% 

3 

16 

MS  Powerpoint 

0 

st  Execution  and  Data  Analysis  (Software  anomaly  detected,  anomaly  documented  in  CPCR,  CPCR  assess 

Engineers  (ND-IV) 

50 

14 

10 

1000 

0% 

50 

665 

1  est  Director  and 

TERMS  are  used  as 
respository 

0 

Document  Results 

Engineers  (ND-IV) 

25 

6 

5 

500 

10% 

50 

665 

Test  Director  and 

TERMS  are  used  as 
respository 

0 

Identify/Resolve  Issues  (Including  assessing  CPCR  for  possible  program  update  and  certification  impact) 

Engineers  (ND-IV) 

25 

15 

10 

0% 

50 

40 

ACSIS 

50000 

High  Cert  Impact  (Yes/No  branch,  No  95%  of  time) 

Engineers  (ND-IV) 

30 

10 

10 

1000 

10% 

50 

8 

ACSIS 

50000 

Conduct  Functional  Area  Assessment  (NO  BRANCH) 

Test  IPT  Lead  (ND-IV) 

15 

11 

10 

1000 

0% 

50 

120 

MS  Excel 

0 

Conduct  Certification  Panel  (NO  BRANCH) 

PM  (ND-V) 

30 

9 

10 

1000 

0% 

1 

16 

MS  Powerpoint 

0 

Correlation  (Ordinal  to  RLT)  0.930399929 

Correlation  (RLT  to  AATP)  0.931962825 

100 

Process  (continued) 

TLT 

Total  Knowledge 

Personel  Cost 

Other  Costs 

Total  Costs 

Price 

Denominator 

Numerator 

ROK 

ROI 

Fleet  Inputs/External  Interface  Requirements/New  System  Resources 

588 

26471 

$  729,385.03 

S  32,612.50 

$  761,997.53 

$  1,276,423.80 

$  761,997.53 

$  647,266.17 

85% 

-15% 

System  Design  Review 

526 

26316 

$  810,427.81 

S  35,125.00 

$  845,552.81 

$  1,418,248.67 

$  845,552.81 

$  643,480.99 

76% 

-24% 

ECP's/SCP's/ICR's 

263 

164474 

$  1,814,390.63 

$  106,250.00 

$  1,920,640.63 

$  3,175,183.59 

$  1,920,640.63 

$  4,021,756.20 

209% 

109% 

Approval  Process  (Including  FPRG,  SCCB,  IWG,  PMS  422) 

982 

2455882 

$  1,357,050.00 

$  75,000.00 

$  1,432,050.00 

$  2,374,837.50 

$  1,432,050.00 

$  60,051,917.22 

4193% 

4093% 

Design  Review  (Including  PDR,  DDWS,  CDR) 

526 

78947 

$  870,907.50 

$  37,000.00 

$  907,907.50 

$  1,524,088.13 

$  907,907.50 

$  1,930,442.97 

213% 

113% 

Design  Walkthrough  (Including  Writing  and  Inspecting  Code,  Unit  Test  and  Analysis,  Debugging  Code) 

1176 

470588 

$  1,548,280.00 

$  48,000.00 

$  1,596,280.00 

$  2,709,490.00 

$  1,596,280.00 

$  11,506,954.20 

721% 

621% 

Develop  Test  Plan  (Including  Test  Specifications  and  Test  Case  Design) 

1053 

105263 

$  4,838,375.00 

$  150,000.00 

$  4,988,375.00 

$  8,467,156.25 

$  4,988,375.00 

$  2,573,923.97 

52% 

-48% 

Test  Procedures 

1053 

105263 

$  4,838,375.00 

$  150,000.00 

$  4,988,375.00 

$  8,467,156.25 

$  4,988,375.00 

$  2,573,923.97 

52% 

-48% 

Test  Readiness  Review 

526 

47368 

$  69,672.60 

$  2,160.00 

$  71,832.60 

$  121,927.05 

$  71,832.60 

$  1,158,265.78 

1612% 

1512% 

st  Execution  and  Data  Analysis  (Software  anomaly  detected,  anomaly  documented  in  CPCR,  CPCR  assess 

1000 

2500000 

$  80,437,984.38 

$  2,493,750.00 

$  82,931,734.38 

$  140,766,472.66 

$  82,931,734.38 

$  61,130,694.18 

74% 

-26% 

Document  Results 

556 

694444 

$  40,218,992.19 

$  1,246,875.00 

$  41,465,867.19 

$  70,383,236.33 

$  41,465,867.19 

$  16,980,748.38 

41% 

-59% 

Identify/Resolve  Issues  (Including  assessing  CPCR  for  possible  program  update  and  certification  impact) 

1000 

1250000 

$  2,419,187.50 

$  125,000.00 

$  2,544,187.50 

$  4,233,578.13 

$  2,544,187.50 

$  30,565,347.09 

1201% 

1101% 

High  Cert  Impact  (Yes/No  branch,  No  95%  of  time) 

1111 

1666667 

$  580,605.00 

$  68,000.00 

S  648,605.00 

$  1,016,058.75 

$  648,605.00 

$  40,753,796.12 

6283% 

6183% 

'  Conduct  Functional  Area  Assessment  (NO  BftAfliH) 

1000 

750000 

$  4,354,537.50 

$  135,000.00 

$  4,489,537.50 

5  7,620,440.63 

$  4,489,537.50 

$  18,339,208.25 

408% 

308% 

Conduct  Certification  Panel  (NO  BRANCH) 

1000 

30000 

$  32,569.20 

$  720.00 

$  33,289.20 

$  56,996.10 

$  33,289.20 

$  733,568.33 

2204% 

2104% 

Totals 

10371684 

$  144,920,739.33 

$  4,705,492.50 

$  149,625,231.83 

5  253,611,293.83 

$  149,626,231.83 

$  253,611,293.83 

TOT 

Personel  Costs 

ND-III 

ND-IV 

ND-V 

Base  Pay 

$  55,585.00 

$  77,414.00 

$  108,564.00 

Location  Adjusted  Hourly  Wage  Contractor  Wage 

$  69,481.25  $  34.74  $  60.80 

$  96,767.50  $  48.38  $  84.67 

$  135,705.00  $  67.85  $  118.74 

Price  Per  Common  Unit  $  24.45 

Table  1 1 .  Decomposed  Off  Ship  Process  (One  Ship) 
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IV.  TO-BE  RESULTS 


A.  “TO  BE”  KVA  ANALYSIS  -  ON  SHIP  AND  OFF  SHIP  AGGREGATE 
PROCESS  MODEL 

The  “To  Be”  analysis  is  a  hypothetical  improved  process  model  of  the  possible 
effects  of  a  future  Open  Architecture  (OA)  enabled  AEGIS  software  maintenance  and 
upgrade  process.  The  processes  and  sub  processes  that  have  been  identified  as  having  the 
potential  for  using  OA  and  Distance  Support  have  been  modified  to  reflect  the 
improvements.  The  potential  improvements  are  described  in  detail  in  the  following 
sections.  The  OA  enabled  processes  and  sub  processes  were  developed  using  current 
distance  support  policy,  Maintenance  Free  Operating  Period  (MFOP)  research,  and 
suggestions  from  subject  matter  experts  (SMEs). 

B.  OPEN  ARCHITECTURE  REENGINEERING 

The  Naval  Surface  Warfare  Center  (NWSC)  Port  Hueneme,  CA  has  been 
developing  the  concept  of  remote  maintenance  of  Cooperative  Engagement  Capability 
(CEC)  on  ships.  Through  the  implementation  of  an  OA  based  system,  the  concept  of 
remote  maintenance  could  greatly  improve  the  AEGIS  software  maintenance  and  upgrade 
process. 

Remote  maintenance  enables  the  Navy  to  provide  distance  support  with 
fielded  systems  and  enhances  the  ability  to  meet  requirements  of  reduced 
manpower,  lower  cost,  and  faster  more  efficient  troubleshooting  and 
repair.  It  works  within  the  initiatives  to  return  ships  to  the  Strike  Group 
faster  and  provides  safe,  effective  and  affordable  combat  systems  into 
future  designs.34 

The  AEGIS  weapons  system  is  already  equipped  with  a  link  to  provide  distance 
support.  “Currently  installed  as  a  fielded  Ship  Alteration  (SHIPALT),  Operational 
Readiness  Test  System  Tech  Assist  Remote  Support  (ORTSTARS)  allows  AEGIS  ships 
to  establish  a  secure  link  between  the  Operational  Readiness  Test  System  (ORTS)  and 

34  NAV SURF WARCENDIV  PORT  HUENEME  CA  2006. 
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any  shore  facility  equipped  with  SIPERNET.”35  This  concept  was  used  in  the 
development  of  the  “To  Be”  model  for  the  AEGIS  software  maintenance  and  upgrade 
process. 

The  Undersea  Warfare  community  has  also  provided  valuable  assistance  through 
their  research  with  MFOP  concept  that  has  been  incorporated  in  the  Acoustic  Rapid 
COTS  Insertion  (ARCI)  program.  The  ARCI  program  incorporated  hardware,  software 
and  COTS  logistics  support  and  technology  to  achieve  a  MFOP  for  90  days.  “An 
additional  benefit  of  the  MFOP  Pilot  Program  was  to  develop  and  implement 
functionality  in  the  ARCI  system  which  further  enables  the  system  to  be  supported  via 
Distance  Support  initiatives.”36  These  important  principals  helped  guide  the 
reengineering  of  the  “To  Be”  model  for  the  AEGIS  software  maintenance  and  upgrade 
process. 

Through  an  analysis  of  the  “As  Is”  process  data,  it  was  determined  that  the  most 
advantageous  improvements  to  the  AEGIS  software  maintenance  and  upgrade  process 
could  be  achieved  through  implementing  the  OA  tenets  of  scalability  and  portability.  The 
analysis  was  concentrated  on  the  Return  on  Investment  (ROI)  findings  from  the  “As  Is” 
process  data,  but  also  heavily  considered  the  concepts  of  distance  support  and,  in  turn, 
Maintenance  Free  Operating  Period  (MFOP).  Reengineering  the  process  provided 
increases  in  the  ROI  for  the  sub  processes  that  were  affected  and  also  produced  an 
increase  in  total  ROI.  Assumptions  for  the  process  reengineering  include: 

*  The  use  of  middleware  was  necessary  until  Category  4  OACE  level  could 
be  reached. 

*  No  process  would  become  fully  100%  automated. 

*  One  employee  would  always  be  on  hand  as  a  supervisor  to  even  a  mostly 
automated  process. 


35  NAV SURF WARCENDIV  PORT  HUENEME  CA  2006. 

36  NAVSEA  Surface  Warfare  Logistics  and  Maintenance  2005. 
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*  The  “Average  Time  to  Complete”  for  the  “New  Software  Version  Fielded 
to  Units”  was  estimated  to  be  15  minutes  using  the  distance  support 
concept. 

*  “Replacement  Technology”  will  be  used  instead  of  “Additive 
Technology.” 

*  Development  costs  were  not  included  because  they  are  distributed 
throughout  the  life  cycle  of  the  system. 

C.  “TO  BE”  PROCESS  DATA 

In  the  following  process  model  and  Knowledge  Value  Added  (KVA)  analysis,  it 
was  necessary  to  estimate  changes  in  the  “Percentage  Automation”  category  due  to  the 
fact  that  the  “To  Be”  model  is  hypothetical.  It  was  also  considered  that  at  least  one 
human  employee  would  oversee  each  of  the  sub  processes  even  though  the  potential  for 
the  sub  processes  to  be  totally  automated  exists. 

In  an  OA  enabled  AEGIS  software  maintenance  and  upgrade  process  software 
updates  can  be  made  available  by  a  push  or  pull  method.  In  the  pull  method,  the  user 
would  download  updates  and  then  install  them.  In  the  push  method,  the  software  would 
be  pushed  to  the  network  node  remotely  therefore  reducing  the  need  for  onsite  personnel. 
These  software  updates  would  take  place  through  the  secure  link  provided  by 
ORTSTARS.  The  ability  of  updates  to  be  made  readily  available  is  an  inherent  benefit  of 
open  architecture  systems  and  serves  to  reduce  the  number  of  personnel  hours  spent  in 
the  AEGIS  software  maintenance  and  upgrade  process.  This  would  also  serve  to 
decrease  the  number  of  on  site  personnel  and  increase  the  speed  of  upgrade. 

The  processes  “Software  Anomaly  Detected,”  “Cause  of  Anomaly  Determined,” 
“Software  Bug  Report  Submitted,”  and  “New  Software  Version  Fielded  to  Units”  were 
determined  to  be  the  most  amenable  to  change  in  the  OA  enabled  method.  In  an  OA 
enabled  AEGIS  environment,  diagnostics  from  a  single  or  multiple  locations  can  be  used 
if  there  is  a  problem  at  the  first  tier  of  support  prior  to  dispatching  personnel.  Eliminating 
on-site  requirements  and  the  need  for  multiple  on  site  maintenance  personnel  will  drive 
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down  costs  and  improve  operational  availability.  These  three  sub  processes  would 
change  into  two  sub  processes;  “Remote  Diagnostics  Detect/Fix  Anomaly”  and  “Remote 
Diagnostics  Submit  Software  Bug  Report  for  Anomaly.”  The  aggregate  processes  that 
were  changed  in  the  OA  enabled  AEGIS  software  maintenance  and  update  process  are 
depicted  in  Table  12. 
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Table  12. 


AEGIS  WEAPONS  SYSTEM 
SOFTWARE  UPDATE 
PROCESS  OPEN 
ARCHITECTURE  (OA) 
ENABLED 


ON  SHIP 


New  Software 
Release  Fielded  to 
Ship  (Pushed  to 
Ship  in  Distance 
Support) 

I 

Remote 

Diagnostics 

Detect/Fix 

Anomaly 


Red  Sub  Processes  indicate 
differences  in  Open  Architecture  (OA) 
enabled  process 


A 


Remote 
Diagnostics 
Submit  Bug  Report 
for  Anomaly 


A 


ANOMALY 

VERIFIED 


_ 

ANOMALY 
APPENDED  TO 
WORKING  LIST 
OF  KNOWN 
ISSUES 


A 


WORKAROUND 

DEVELOPED 


OFF  SHIP  -  NSWC  DAHLGREN,  VA 


SOFTWARE 

ANOMALY 

DISCOVERED 


OA  enabled  AEGIS  software  maintenance  and  upgrade  process  (Aggregate  level) 
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1. 


Remote  Diagnostics  Detect/Fix  Anomaly 


The  transfer  to  OA  in  new  systems  and  converting  to  OA  is  old,  closed  systems 
enables  the  use  of  remote  diagnostics  in  the  “To  Be”  process  model.  Through 
ORTSTARS,  a  remote  diagnostic  can  identify  a  software  anomaly  potentially  before  an 
operator  on  the  ship  can  identify  the  anomaly.  The  remote  diagnostics  then  could  record 
the  circumstances  surrounding  the  anomaly  and  compare  them  to  similar  Computer 
Program  Change  Requests  (CPCRs)  managed  in  the  ACCESS/STARSY  database.  If  a 
CPCR  is  found  that  closely  matches  the  anomaly  detected,  the  remote  diagnostics  could 
then  take  appropriate  actions  already  listed  in  the  ACCESS/STARSY  database  to  fix  the 
anomaly. 

The  increase  in  remote  diagnostics  capabilities  through  the  implementation  of  OA 
would  drastically  reduce  the  number  of  personnel  required  to  complete  the  process.  In 
the  “As  Is”  model  the  “Remote  Diagnostics  Detect/Fix  Anomaly”  was  two  separate 
processes;  “Software  Anomaly  Detected”  and  “Cause  of  Anomaly  Determined”. 
Distance  Support  allowed  these  two  processes  to  be  combined  into  one  process  and 
reduced  the  number  of  personnel  required  to  complete  the  processes  from  seven 
employees  to  one  employee.  This  allows  for  the  process  to  become  90%  automated 
while  still  preserving  some  human  intervention. 

2.  Remote  Diagnostics  Submit  Software  Bug  Report  for  Anomaly 

The  usage  of  OA  allows  for  the  use  of  remote  diagnostics  to  submit  a  software 
bug  report.  Again  through  ORTSTARS,  the  software  bug  report  could  be  submitted 
through  a  secure  link  in  real  time,  and  the  software  bug  report  has  potential  to  be  a  more 
thorough  representation  of  the  circumstances  surrounding  the  anomaly  as  no  human 
interpretation  would  be  required.  The  process  still  retains  some  human  intervention  as 
one  process  executor  would  still  oversee  the  process.  This  combination  of  OA  and 
remote  diagnostics  could  also  drastically  reduce  cycle  time  in  submitting  the  software 
bug  report. 
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3. 


New  Software  Version  Fielded  to  Units  (Pushed  to  Ship  via  Distance 
Support) 


In  the  OA  enabled  model  for  the  AEGIS  software  maintenance  and  upgrade 
process,  new  software  updates  could  be  fielded  to  the  ship  through  ORTSTARS  in  either 
the  push  method  or  the  pull  method.  This  would  allow  for  greatly  reduced  cycle  time  in 
the  fielding  of  new  software  in  its  shipboard  configuration.  Remote  diagnostics  could 
also  perform  the  functions  involved  in  the  “Combat  System  Integration  Test”  further 
reducing  cycle  time.  Software  fielding  through  distance  support  and  the  push/pull 
method  would  also  reduce  the  number  of  personnel  required  to  field  the  software  to  the 
unit  from  three  employees  to  one  employee.  The  one  process  executor  would  still  remain 
on  hand  to  oversee  the  process  and  resolve  any  issues,  via  distance  support,  that  the  ship 
may  encounter  once  the  software  has  been  fielded  in  its  shipboard  configuration. 

4.  OA  Enabled  On  and  Off  Ship  Aggregate  Process 

Table  13  depicts  the  OA  enabled  on  and  off  ship  process  included  in  the  KVA 
analysis  for  the  OA  enabled  AEGIS  software  maintenance  and  upgrade  process  that 
occurs  for  one  AEGIS  ship. 
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Process 

Title  of  Head  Process  Executer 

Number  of  Employees 

Rank  Order  of  Difficulty 

Relative  Learning 
Time 

Actual  Average 
Learning  Time 

Percentage 

Automation 

Times  Performed  In  a 

Year 

Average  Time  to 
Complete 

Automation  Tools 

New  Release  Fielded  (Pushed  to  Ship  via  Distance  Support) 

Fleet  Support  (GS-9/10) 

1 

4 

10 

300 

90% 

10 

20 

N/A 

Remote  Diagnostics  Detect/Fix  Anomaly 

Technology  Director  (GS-12/13) 

i 

8 

20 

500 

95% 

25 

40 

Advanced  Software 

Remote  Diagnostics  Submit  Software  Bug  Report  for  Anomaly 

Project  Manager  (GS-11) 

i 

3 

10 

300 

95% 

10 

4 

MS  Word 

Anomaly  Verified 

Project  Manager  (GS-1 1/12) 

2 

5 

10 

300 

0% 

10 

20 

N/A 

Anomaly  Appended  to  Working  List  of  Known  Issues 

Fleet  Support  (GS-9/10) 

1 

2 

5 

160 

10% 

10 

4 

Excel 

Workaround  Developed 

Project  Manager  (GS-1 1/12) 

2 

6 

10 

160 

0% 

10 

20 

N/A 

New  Software  Version  Developed 

Lead  Programmer  (GS-1 3/14) 

10 

9 

20 

640 

20% 

2 

200 

Compiler 

Known  Anomalies  are  Resolved 

Lead  Programmer  (GS-12/13) 

5 

7 

10 

500 

20% 

2 

200 

N/A 

New  Software  Version  Fielded  to  Units  (Pushed  to  Ship  via  Distance  Support) 

Fleet  Support  (GS-9/10) 

1 

i 

5 

160 

90% 

20 

0.25 

Tracking 

Correlation  (Ordinal  to  RLT)  0.877032523 

Correlation  (RLT  to  AATP)  0.845793835 


Process  (continued) 

TLT 

Total  Knowledge 

Personel  Cost 

Other  Costs 

Total  Costs 

Price 

Denominator 

Numerator 

ROK 

ROI 

New  Release  Fielded  (Push  to  Ship  via  Distance  Support) 

3000 

30000 

$  6,850.50 

$  300.00 

$  7,150.50 

$  10,911.25 

7150.5 

284131.3652 

3974% 

3874% 

Remote  Diagnostics  Detect/Fix  Anomaly 

10000 

250000 

$  48,819.38 

t  1,500.00 

$  50,319.38 

$  85,433.91 

50319.375 

2367761.377 

4705% 

4605% 

Remote  Diagnostics  Submit  Software  Bug  Report  for  Anomaly 

6000 

60000 

$  1,370.10 

$  60.00 

$  1,430.10 

$  2,397.68 

1430.1 

568262.7304 

39736% 

39636% 

Anomaly  Verified 

300 

6000 

$  16,421.50 

$  600.00 

$  17,021.50 

$  28,737.63 

17021.5 

56826.27304 

334% 

234% 

Anomaly  Appended  to  Working  List  of  Known  Issues 

178 

1778 

$  1,247.00 

t  60.00 

$  1,307.00 

$  2,182.25 

1307 

16837.41424 

1288% 

1188% 

Workaround  Developed 

160 

3200 

$  16,421.50 

$  600.00 

$  17,021.50 

$  28,737.63 

17021.5 

30307.34562 

178% 

78% 

New  Software  Version  Developed 

800 

16000 

$  230,750.00 

{  6,000.00 

$  236,750.00 

$  403,812.50 

236750 

151536.7281 

64% 

■36% 

Known  Anomalies  are  Resolved 

625 

6250 

$  97,638.75 

{  3,000.00 

$  100,638.75 

$  170,867.81 

100638.75 

59194.03442 

59% 

■41% 

New  Software  Version  Fielded  to  Units  (Pushed  to  Ship  via  Distance  Support) 

1600 

32000 

$  155.88 

1  tst 

$  163.38 

$  272.78 

163.375 

303073.4562 

185508% 

185408% 

Totals 

405228 

T  419,674.60 

1  1242750 

1  431,802.10 

1  733,353.43 

431802.1 

3837930.724 

889% 

wr~ 

Personel  Costs 

Base  Pay 

Location  Adjusted 

Hourly  Wage  Contractor  Wage 

GS9 

$ 

45,294.00  $ 

56,617.50  3 

28.31  $ 

49.54 

GS10 

$ 

49,880.00  3 

62,350.00  5 

31.18  $ 

54.56 

GS11 

$ 

54,804.00  3 

68,505.00  3 

34.25  3 

59.94 

GS12 

$ 

65,686.00  3 

82,107.50  3 

41.05  3 

71.84 

GS13 

$ 

78,111.00  3 

97,638.75  3 

48.82  $ 

85.43 

GSM 

$ 

92,300.00  3 

115,375.00  3 

57.69  $ 

100.95 

Price  Per  Common  Unit 

3 

9.47 

Table  13.  KVA  OA  Enabled  (One  Ship) 
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5.  OA  Enabled  On  and  Off  Ship  Aggregate  Process  for  Entire  AEGIS 
Fleet 

Table  14  depicts  the  further  decomposed  off  ship  process  included  in  the  KVA 
analysis  for  the  AEGIS  software  maintenance  and  upgrade  process  that  is  scaled  to 
include  all  AEGIS  ships  in  the  U.S.  Fleet.  Assumptions  for  scaling  the  process  data 
include: 

o  Each  software  update  is  fielded  to  each  of  the  84  AEGIS  ships  on  a  case 
by  case  basis. 

o  Anomalies  are  resolved  on  a  case  by  case  basis. 
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Process 

Title  of  Head  Process  Executer 

Number  of  Employees 

Rank  Order  of  Difficulty 

Relative  Learning 
Time 

Actual  Average 
Learning  Time 

Percentage 

Automation 

Times  Performed  In  a 

Year 

Average  Time  to 
Complete 

Automation  Tools 

New  Release  Fielded  (Pushed  to  Ship  via  Distance  Support) 

Fleet  Support  (GS-9/10) 

1 

4 

10 

300 

90’/. 

10 

20 

N/A 

Remote  Diagnostics  Detect/Fix  Anomaly 

Technology  Director  (GS-12/13) 

1 

8 

20 

500 

95% 

25 

40 

Advanced  Software 

Remote  Diagnostics  Submit  Software  Bug  Report  for  Anomaly 

Project  Manager  (GS-1 1 ) 

1 

3 

10 

300 

95% 

10 

4 

MS  Word 

Anomaly  Verified 

Project  Manager  (GS-1 1/12) 

2 

5 

10 

300 

0% 

10 

20 

N/A 

Anomaly  Appended  lo  Working  List  of  Known  Issues 

Fleet  Support  (GS-9/10) 

1 

2 

5 

160 

10% 

10 

4 

Excel 

Workaround  Developed 

Project  Manager  (GS-11/12) 

2 

6 

10 

160 

0% 

10 

20 

N/A 

New  Software  Version  Developed 

Lead  Programmer  (GS-13/14) 

10 

9 

20 

640 

20% 

2 

200 

Compiler 

Known  Anomalies  are  Resolved 

Lead  Programmer  (GS-12/13) 

5 

7 

10 

500 

20% 

2 

200 

N/A 

New  Software  Version  Fielded  to  Units  (Pushed  lo  Ship  via  Distance  Support) 

Fleet  Support  (GS-9/10) 

1 

1 

5 

160 

90% 

20 

0.25 

Tracking 

Correlation  (Ordinal  to  RLT)  0.877032523 

Correlation  (RLT  to  MTP)  0.845793835 


Process  (continued) 

TLT 

Total  Knowledge 

Personel  Cost 

Other  Costs 

Total  Costs 

Price 

Denominator 

Numerator 

ROK 

ROI 

New  Release  Fielded  (Push  to  Ship  via  Distance  Support) 

3000 

30000 

$  6,850.50 

$  300.00 

$  7,150.50 

$  10,911.25 

7150.5 

23867034.68 

333781% 

333681% 

Remote  Diagnostics  Detect/Fix  Anomaly 

10000 

250000 

$  48,819.38 

3  1,500.00 

$  50,319.38 

$  85,433.91 

50319.375 

198891955.7 

395259% 

395159% 

Remote  Diagnostics  Submit  Software  Bug  Report  for  Anomaly 

6000 

60000 

$  1,370.10 

3  60.00 

3  1,430.10 

$  2,397.68 

1430.1 

47734069.36 

3337813% 

3337713% 

Anomaly  Verified 

300 

6000 

$  16,421.50 

(  600.00 

$  17,021.50 

$  28,737.63 

17021.5 

4773406.936 

28043% 

27943% 

Anomaly  Appended  lo  Working  List  of  Known  Issues 

178 

1778 

$  1,247.00 

3  60.00 

$  1,307.00 

$  2,182.25 

1307 

1414342.796 

108213% 

108113% 

Workaround  Developed 

160 

3200 

$  16,421.50 

3  600.00 

$  17,021.50 

$  28,737.63 

17021.5 

2545817.032 

14956% 

14856% 

New  Software  Version  Developed 

800 

16000 

$  230,750.00 

(  6,000.00 

$  236,750.00 

$  403,812.50 

236750 

12729085.16 

5377% 

5277% 

Known  Anomalies  are  Resolved 

625 

6250 

$  97,638.75 

3  3,000.00 

$  100,638.75 

$  170,867.81 

100638.75 

4972298.891 

4941% 

4841% 

New  Software  Version  Fielded  to  Units  (Pushed  lo  Ship  via  Distance  Support) 

1600 

32000 

$  155.88 

(  7.50 

$  13,723.50 

$  272.78 

13723.5 

25458170.32 

185508% 

185408% 

Totals 

1  4196® 

]  iW50 

1  44566223 

1  73365343 

445362.225 

322386180.8 

72387% 

72287% 

Personel  Costs 


Location  Adjusted 


Hourly  Wage  Contractor  Wage 


GS9 

$ 

45,294.00  $ 

56,617.50  $ 

28.31  $ 

49.54 

GS10 

$ 

49,880.00  $ 

62,350.00  $ 

31.18  $ 

54.56 

GS11 

$ 

54,804.00  $ 

68,505.00  $ 

34.25  $ 

59.94 

GS12 

$ 

65,686.00  $ 

82,107.50  $ 

41.05  $ 

71.84 

GS13 

« 

78,111.00  $ 

97,638.75  $ 

48.82  $ 

85.43 

GS14 

* 

92,300.00  $ 

115,375.00  $ 

57.69  $ 

100.95 

Price  Per  Common  Unit 

$ 

9.47 

Table  14.  KVA  OA  Enabled  (All  Ships) 
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6. 


“To  Be”  Process  Data  Analysis 


The  “To  Be”  OA  enabled  model  and  the  implementation  of  Distance  Support  and 
Monitoring  for  the  AEGIS  software  maintenance  and  upgrade  process  has  produced 
appreciable  increases  in  the  ROI  for  each  of  the  sub  processes.  Each  of  the  sub  processes 
that  were  changed  through  the  OA  transformation  experienced  an  increase  in  the 
categories  “Total  Knowledge”  and  the  “Numerator.”  The  “Numerator”  category 
represents  revenue  for  each  of  the  sub  processes. 

The  increase  in  “Total  Knowledge”  was  due  to  several  factors.  The  OA  enabled 
AEGIS  software  maintenance  and  upgrade  process  allows  for  easier  anomaly 
identification.  Once  the  anomaly  is  identified,  the  OA  enabled  process  allows  for  a  more 
complete  representation  of  the  circumstances  surrounding  the  anomaly.  Both  of  these 
factors  allow  for  an  increase  in  the  knowledge,  in  hours,  over  a  given  year  using  the  OA 
enabled  AEGIS  software  maintenance  and  upgrade  process.  The  increase  in  “Total 
Knowledge”  was  also  affected  by  a  remote  diagnostics  network  that  could  make  the 
anomaly  information  available  to  a  subject  matter  expert  rather  than  being  assessed  solely 
by  ship  personnel.  The  remote  diagnostics  allow  for  easier  collaboration  between  SME’s 
and  personnel  on  the  ship. 

The  category  “Numerator”  was  also  substantially  increased.  This  increase  in 
revenue  was  primarily  due  to  more  units  of  knowledge  being  made  available  at  the  same 
price  per  unit  of  knowledge.  The  increase  in  revenue  was  due  to  a  more  efficient  OA 
enabled  AEGIS  software  maintenance  and  upgrade  process.  The  move  to  OA  facilitated 
the  use  of  distance  support  and  allowed  for  improvements  to  key  sub  processes  that 
changed  the  current  “As  Is”  process  to  an  entirely  different  process.  The  “To  Be”  OA 
enabled  AEGIS  software  maintenance  and  upgrade  process  provides  a  more  efficient, 
highly  automated  alternative  to  the  current  “As  Is”  process 

The  increases  in  the  category  “Numerator”  or  revenue,  “Total  Knowledge,” 

“ROK”  and  “ROI”  in  the  OA  enabled  AEGIS  software  maintenance  and  upgrade  process 

were  estimated  using  a  conservative  method.  The  potential  for  further  increases  in  each 

of  the  categories  mentioned  above  exists,  but  is  very  difficult  to  properly  document.  For 
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the  purposes  of  this  research,  the  category  “Average  Time  to  Complete”  was  left 
unchanged,  except  for  the  process  “New  Software  Version  Fielded  to  Units.”  The 
“Average  Time  to  Complete”  this  process  was  estimated  to  be  15  minutes  in  the  “To  Be” 
model.  This  estimation  was  based  on  SME  inputs  along  with  precedents  that  were 
established  in  distance  support  policy.  The  other  processes  in  the  AEGIS  software 
maintenance  and  upgrade  process  have  the  potential  for  decreased  completion  times. 
Implementing  remote  station  monitoring  allows  employees  to  become  more  efficient  and 
potentially  decrease  the  “Average  Time  to  Complete”  in  the  execution  of  each  of  their 
processes.  Since  there  is  no  way  to  accurately  project  or  estimate  this  increase  of 
efficiency,  this  was  not  taken  into  account  in  this  research. 

Due  to  the  fact  that  the  sub  processes  changed  in  the  OA  enabled  AEGIS  software 
maintenance  and  upgrade  process,  the  process  executors  would  need  to  relearn  how  to 
execute  each  of  the  sub  processes.  Even  though  training  can  facilitate  this  learning  a 
substantial  amount  of  learning  still  takes  place  on  the  job  or  “learning  by  doing.”  It 
would  be  expected  that  over  time  the  sub  processes  would  become  even  more  efficient 
than  represented  in  this  analysis.  Further  increased  efficiency  would  serve  to  both 
decrease  cycle  time  and  also  decrease  the  “Average  Time  to  Complete”  for  each  of  the 
sub  processes.  This  could  not  be  projected  in  this  work  due  to  data  collection  challenges, 
as  the  amount  of  efficiency  increase  cannot  be  accurately  predicted. 

D.  COMPARATIVE  ANALYSIS 

Now  that  both  the  “As  Is”  and  “To  Be”  process  models  and  data  have  been 
presented,  it  is  valuable  to  present  them  in  a  side  by  side  comparison.  Each  of  the  sub 
processes  are  presented  with  their  corresponding  ROI’s  for  both  the  “As  Is”  and  “To  Be” 
configurations. 
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"As  Is' 


Process 

Revenue 

Total  Costs 

ROI 

Software  Anomaly  Detected 

$ 

56,826.27 

$ 

14,301.00 

297% 

Cause  of  Anomaly  Determined 

$ 

845,629.06 

$ 

251,596.88 

236% 

Software  Bug  Report  Submitted 

$ 

31,570.15 

$ 

1,430.10 

2108% 

Anomaly  Verified 

$ 

56,826.27 

$ 

17,021.50 

234% 

Anomaly  Appended  to  Working  List  of  Known  Issues 

$ 

16,837.41 

$ 

1,307.00 

1188% 

Workaround  Developed 

$ 

30,307.35 

$ 

17,021.50 

78% 

New  Software  Version  Developed 

$ 

151,536.73 

$ 

236,750.00 

-36% 

Known  Anomalies  are  Resolved 

$ 

59,194.03 

$ 

100,638.75 

-41% 

New  Software  Version  Fielded  to  Units 

$ 

101,024.49 

$ 

156,840.00 

-36% 

Totals 

$1,349,751.77 

$ 

796,906.73 

69% 

"To  Be" 


Process 

Revenue 

Total  Costs 

ROI 

New  Release  Fielded  (Push  to  Ship  via  Distance  Support) 

$  284,131.37 

$ 

7,150.50 

3874% 

Remote  Diagnostics  Detect/Fix  Anomaly 

$2,367,761.38 

$ 

50,319.38 

4605% 

Remote  Diagnostics  Submit  Software  Bug  Report  for  Anomaly 

$  568,262.73 

$ 

1,430.10 

39636% 

Anomaly  Verified 

$  56,826.27 

$ 

17,021.50 

234% 

Anomaly  Appended  to  Working  List  of  Known  Issues 

$  16,837.41 

$ 

1,307.00 

1188% 

Workaround  Developed 

$  30,307.35 

$ 

17,021.50 

78% 

New  Software  Version  Developed 

$  151,536.73 

$ 

236,750.00 

-36% 

Known  Anomalies  are  Resolved 

$  59,194.03 

$ 

100,638.75 

-41% 

New  Software  Version  Fielded  to  Units  (Pushed  to  Ship  via  Distance  Support) 

$  303,073.46 

$ 

163.38 

185408% 

Totals 

$3,837,930.72 

$ 

431,802.10 

789% 

Table  15.  Side  by  Side  Comparison  (One  Ship) 


The  side  by  side  comparison  of  the  “As  Is”  and  “To  Be”  models  shown  above 
demonstrate  the  dramatic  effect  that  OA  and  distance  support  initiatives  could  have  on 
the  AEGIS  software  maintenance  and  upgrade  process.  The  increase  in  revenue  by 
$2,488,178.96,  the  cost  savings  of  $365,104.63,  and  the  increase  in  ROI  from  69%  to 
720%  after  reengineering  the  AEGIS  software  maintenance  and  upgrade  process  using  an 
OA  approach  depicts  a  substantial  increase  in  efficiency  of  each  of  the  affected  sub 
processes  and  also  the  process  as  a  whole.  This  side  by  side  comparison  represents  the 
efficiency  improvements  for  one  ship  in  the  current  AEGIS  fleet.  It  is  also  of  value  to 
present  the  side  by  side  comparisons  for  the  entire  AEGIS  fleet. 


65 


'As  Is1 


Process 

Revenue 

Total  Costs 

ROI 

Software  Anomaly  Detected 

$ 

4,773,406.94 

$ 

14,301.00 

33278% 

Cause  of  Anomaly  Determined 

$ 

71,032,841.31 

$ 

251,596.88 

28133% 

Software  Bug  Report  Submitted 

$ 

2,651,892.74 

$ 

1,430.10 

185334% 

Anomaly  Verified 

$ 

4,773,406.94 

$ 

17,021.50 

27943% 

Anomaly  Appended  to  Working  List  of  Known  Issues 

$ 

1,414,342.80 

$ 

1,307.00 

108113% 

Workaround  Developed 

$ 

2,545,817.03 

$ 

17,021.50 

14856% 

New  Software  Version  Developed 

$ 

12,729,085.16 

$ 

236,750.00 

5277% 

Known  Anomalies  are  Resolved 

$ 

4,972,298.89 

$ 

100,638.75 

4841% 

New  Software  Version  Fielded  to  Units 

$ 

8,486,056.77 

$ 

26,349,120.00 

-68% 

Totals 

$  113,379,148.58 

$ 

26,989,186.73 

320% 

"To  Be" 


Process 

Revenue 

Total  Costs 

ROI 

New  Release  Fielded  (Push  to  Ship  via  Distance  Support) 

$ 

23,867,034.68 

$ 

7,150.50 

333681% 

Remote  Diagnostics  Detect/Fix  Anomaly 

$  198,891,955.66 

$ 

50,319.38 

395159% 

Remote  Diagnostics  Submit  Software  Bug  Report  for  Anomaly 

$ 

47,734,069.36 

$ 

1,430.10 

3337713% 

Anomaly  Verified 

$ 

4,773,406.94 

$ 

17,021.50 

27943% 

Anomaly  Appended  to  Working  List  of  Known  Issues 

$ 

1,414,342.80 

$ 

1,307.00 

108113% 

Workaround  Developed 

$ 

2,545,817.03 

$ 

17,021.50 

14856% 

New  Software  Version  Developed 

$ 

12,729,085.16 

$ 

236,750.00 

5277% 

Known  Anomalies  are  Resolved 

$ 

4,972,298.89 

$ 

100,638.75 

4841% 

New  Software  Version  Fielded  to  Units  (Pushed  to  Ship  via  Distance  Support) 

$ 

25,458,170.32 

$ 

13,723.50 

185408% 

Totals 

$  322,386,180.83 

$ 

445,362.23 

72287% 

Table  16.  Side  by  Side  Comparison  (All  Ships) 


The  side  by  side  comparison  of  the  “As  Is”  and  “To  Be”  models  shown  above  for 
all  AEGIS  ships  in  the  U.S.  fleet  demonstrate  the  dramatic  effect  that  OA  and  distance 
support  initiatives  could  have  on  the  AEGIS  software  maintenance  and  upgrade  process. 
The  increase  in  revenue  by  $209,007,032.26,  the  cost  savings  of  $26,543,824.50,  and  the 
increase  in  ROI  by  71987%  represent  the  substantial  benefits  that  can  be  achieved  when 
the  OA  enabled  AEGIS  software  maintenance  and  upgrade  process  is  applied  to  all 
AEGIS  ships. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS  AND  RECOMMENDATIONS 

Proprietary  closed  architecture  systems,  such  as  the  AEGIS  system,  have  been 
effective  systems  that  have  provided  the  Navy  with  important  operational  capabilities  in 
the  past.  As  these  systems  age,  there  becomes  a  need  for  increased  Sustaining 
Engineering  (SE)  support  for  these  systems.  The  Component  Business  Model  (CBM) 
conducted  by  IBM  and  the  large  investment  in  SE  reaffirmed  the  importance  and  need  for 
efficiency.  This  is  especially  evident  in  the  AEGIS  software  maintenance  and  upgrade 
process.  The  current  proprietary,  closed  architecture  design  of  the  AEGIS  system  makes 
the  AEGIS  software  maintenance  and  upgrade  process  a  very  costly  and  time  intensive 
process  requiring  a  great  deal  of  personnel.  Incompatibility  and  missed  opportunities  for 
new  technologies  are  a  considerable  problem  for  the  AEGIS  software  maintenance  and 
upgrade  process  and  for  acquirers  and  developers. 

The  incorporation  of  Open  Architecture  (OA)  would  allow  current  proprietary 
systems  to  leverage  new  technologies  in  an  effort  to  increase  efficiency  and  realize  the 
full  potential  of  the  Navy’s  systems  and  processes.  Current  programs  and  policies  such 
as  distance  support  and  Maintenance  Free  Operating  Period  (MFOP)  could  be  easily 
integrated  into  the  AEGIS  software  maintenance  and  upgrade  process  through  a  move  to 
OA.  This  thesis  provided  insight  into  the  operational  value  that  can  be  achieved  by  using 
an  OA  framework  in  the  AEGIS  software  maintenance  and  upgrade  process  through  an 
increase  in  Return  on  Investment  (ROI). 

The  current  AEGIS  software  maintenance  and  upgrade  process  is  a  fairly  efficient 
process.  The  total  ROI  associated  with  the  “As  Is”  AEGIS  software  maintenance  and 
upgrade  process  is  69%.  This  indicates  that  the  process  returns  more  revenue  than  the 
cost  of  the  aggregate  process;  however,  even  though  the  “As  Is”  process  produces  a 
positive  ROI,  the  potential  for  increased  ROI  exists  through  the  transition  to  an  OA 
enabled  AEGIS  software  maintenance  and  upgrade  process. 
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The  “To  Be”  OA  enabled  AEGIS  software  maintenance  and  upgrade  process 
incorporates  the  OA  tenets  of  scalability  and  portability.  The  OA  framework  allows  for 
several  of  the  sub  processes  to  be  changed  to  allow  for  the  use  of  distance  support  and 
MFOP  concepts.  These  concepts  enabled  decreased  personnel  and  increased  automation 
in  the  processes  “New  Software  Fielded  to  Units,”  “Remote  Diagnostics  Detect/Fix 
Anomaly,”  and  “Remote  Diagnostics  Submit  Software  Bug  Report  for  Anomaly.”  The 
improvements  made  in  the  “To  Be”  OA  enabled  AEGIS  software  maintenance  and 
upgrade  process  greatly  increased  the  ROI  for  each  of  the  improved  sub  processes  and 
also  for  the  entire  process  as  a  whole.  The  total  ROI  and  costs  saving  associated  with  the 
“To  Be”  OA  enabled  AEGIS  software  maintenance  and  upgrade  process  is  720%  and 
$365,104.10.  This  represents  a  sizable  improvement  in  efficiency  with  the  incorporation 
of  OA  in  the  AEGIS  software  maintenance  and  upgrade  process.  The  improvement  in 
efficiency  is  even  more  apparent  when  it  is  applied  to  all  AEGIS  ships  in  the  current  U.S. 
Navy  fleet.  This  can  be  seen  below  in  Table  18. 


"As  Is" 


Process 

Revenue 

Total  Costs 

ROI 

Software  Anomaly  Detected 

$ 

56,826.27 

$ 

14,301.00 

297% 

Cause  of  Anomaly  Determined 

$ 

845,629.06 

$ 

251,596.88 

236% 

Software  Bug  Report  Submitted 

$ 

31,570.15 

$ 

1,430.10 

2108% 

Anomaly  Verified 

$ 

56,826.2 7 

$ 

17,021.50 

234% 

Anomaly  Appended  to  Working  List  of  Known  Issues 

$ 

16,837.41 

$ 

1,307.00 

1188% 

Workaround  Developed 

$ 

30,307.35 

$ 

17,021.50 

78% 

New  Software  Version  Developed 

$ 

151,536.73 

$ 

236,750.00 

-36% 

Known  Anomalies  are  Resolved 

$ 

59,194.03 

$ 

100,638.75 

-41% 

New  Software  Version  Fielded  to  Units 

$ 

101,024.49 

$ 

156,840.00 

-36% 

Totals 

$1,349,751.77 

i 

796,906.73 

69% 

"To  Be" 


Process 

Revenue 

Total  Costs 

ROI 

New  Release  Fielded  (Push  to  Ship  via  Distance  Support) 

$ 

284,131.37 

$ 

7,150.50 

3874% 

Remote  Diagnostics  Detect/Fix  Anomaly 

$2,367,761.38 

$ 

50,319.38 

4605% 

Remote  Diagnostics  Submit  Software  Bug  Report  for  Anomaly 

$ 

568,262.73 

$ 

1,430.10 

39636% 

Anomaly  Verified 

$ 

56,826.27 

$ 

17,021.50 

234% 

Anomaly  Appended  to  Working  List  of  Known  Issues 

$ 

16,837.41 

$ 

1,307.00 

1188% 

Workaround  Developed 

$ 

30,307.35 

$ 

17,021.50 

78% 

New  Software  Version  Developed 

$ 

151,536.73 

$ 

236,750.00 

-36% 

Known  Anomalies  are  Resolved 

$ 

59,194.03 

$ 

100,638.75 

-41% 

New  Software  Version  Fielded  to  Units  (Pushed  to  Ship  via  Distance  Support) 

$ 

303,073.46 

$ 

163.38 

185408% 

Totals 

$3,837,930.72 

$ 

431,802.10 

789% 

Table  17.  Side  by  Side  Comparison  (One  Ship) 


There  are  several  potential  benefits  that  are  not  captured  in  the  KVA  analysis 
performed  in  this  research.  The  increase  in  capabilities  that  occurs  from  the  remote 
maintenance  and  upgrade  ability  of  the  OA  enabled  AEGIS  software  maintenance  and 
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upgrade  process  was  not  captured  in  the  KVA  analysis.  This  would  allow  for  increased 
capabilities  to  be  delivered  to  the  warfighter.  These  increased  capabilities  could  allow 
the  warfighter  to  maintain  a  more  complete  operational  picture  and  lead  to  increased 
operational  effectiveness. 

The  OA  enabled  AEGIS  software  maintenance  and  upgrade  process  allows  for  the 
software  to  be  delivered  to  operational  units  much  sooner  than  the  “As  Is”  configuration. 
Due  to  this,  it  is  important  to  consider  the  concept  of  the  time  value  of  money.  The  time 
value  of  money  concept  suggests  that  a  dollar  delivered  today  would  be  worth  more  than 
a  dollar  delivered  in  a  month  from  today.  This  same  principle  can  be  applied  to  software 
updates  and  operational  capabilities.  The  sooner  that  a  software  update  can  be  fielded, 
the  greater  benefit  it  can  provide  to  the  operational  unit  and  the  warfighter. 

B.  RESEARCH  LIMITATIONS 

The  data  collected  and  analyzed  in  this  research  was  provided  by  subject  matter 
experts  (SME’s).  These  SME’s  each  have  a  different  background  and  current  level  of 
experience.  These  factors  have  an  effect  on  the  data  and  suggestions  provided  by  each 
SME.  This  gives  some  level  of  subjectivity  to  the  data  that  was  collected.  Until  data 
capturing  methods  are  in  place  to  collect  historical  data  associated  with  the  AEGIS 
software  maintenance  and  upgrade  process,  SME  inputs  will  continue  to  be  the  main 
method  for  data  collection  in  this  type  of  analysis. 

The  “To  Be”  analysis  was  based  on  inputs  from  SME’s  as  to  the  best  and  most 
feasible  areas  to  implement  OA  and  distance  support  initiatives.  Inputs  from  programs 
implemented  in  other  communities,  such  as  the  Maintenance  Free  Operating  Period 
utilized  by  the  Undersea  Warfare  Community,  were  carefully  considered  but  ultimately 
SME  inputs  detennined  what  was  feasible  for  the  AEGIS  software  maintenance  and 
upgrade  process.  The  “To  Be”  process  provides  a  conceptual  framework  for  the  process 
reengineering  of  the  AEGIS  software  maintenance  and  upgrade  process  without  taking 
into  account  technical  and  legal  aspects  associated  with  the  reengineering. 

The  KVA  analysis  perfonned  in  this  research  estimated  the  reduction  in  costs  that 

would  occur  through  transformation  to  an  OA  enabled  AEGIS  software  maintenance  and 
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upgrade  process.  Due  to  data  collection  challenges,  costs  were  estimated  based  on  the 
number  of  employees  and  the  cost  of  information  technology  associated  with  each  sub 
process  in  the  AEGIS  software  maintenance  and  upgrade  process.  The  reduction  in  cost 
that  occurred  in  the  transformation  to  the  OA  enabled  AEGIS  software  maintenance  and 
upgrade  process  was  most  likely  underestimated.  Remote  monitoring  enables  reductions 
in  personnel  and  work  time  that  were  accounted  for  in  the  KVA  analysis.  The  KVA 
analysis  did  not  account  for  the  cost  savings  to  Sustaining  Engineering  (SE)  activities  not 
specifically  associated  with  the  AEGIS  software  maintenance  and  upgrade  process. 
Processes  outside  the  AEGIS  software  maintenance  and  upgrade  process  but  still 
included  in  SE  such  as  configuration  management,  verification  and  validation  could 
represent  additional  cost  savings  through  the  transformation  to  OA. 
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VI.  FUTURE  RESEARCH 


A.  FUTURE  RESEARCH 

This  thesis  explored  the  possibility  of  implementing  an  Open  Architecture  (OA) 
approach  to  the  AEGIS  software  maintenance  and  upgrade  process.  The  reengineering  of 
the  AEGIS  software  maintenance  and  upgrade  process  using  OA  showed  that  there  are 
definite  benefits  to  be  gained  through  implementing  the  principles  of  OA  into  system  and 
process  design.  The  move  to  an  Open  Architecture  framework  is  a  large  undertaking  that 
will  require  a  considerable  change  in  thinking  when  designing  new  system  interfaces  and 
ways  to  bridge  existing  proprietary  closed  architecture  systems  to  open  systems.  The 
Navy  has  recognized  the  benefits  of  Open  Architecture  and  through  the  work  of  the 
Project  Executive  Office  Integrated  Warfare  Systems  (PEO  IWS)  will  continue  to 
improve  existing  legacy  systems. 

A  baseline  has  been  set  in  this  research  for  the  value  of  integrating  OA  into  the 
AEGIS  software  maintenance  and  upgrade  process.  There  is  still  much  research  that  can 
be  conducted  to  evaluate  the  benefit  of  OA  in  the  AEGIS  software  maintenance  and 
upgrade  process.  A  great  possibility  exists  to  explore  the  impact  of  OA  on  ship  logistics. 
This  research  reinforced  the  fact  that  OA  enables  personnel  reductions  and  decreased 
cycle  time.  These  benefits  free  up  time  for  operators  on  the  ship  to  focus  on  more 
mission  critical  areas  rather  than  Sustaining  Engineering  (SE)  processes.  The  benefits  of 
this  increase  on  operational  mission  areas  could  positively  affect  operational  efficiency 
and  could  lead  to  additional  research  areas.  There  is  also  the  possibility  of  including 
increased  capabilities  into  each  software  upgrade  that  is  fielded.  The  decreased  cycle 
time  that  is  produced  by  the  OA  enabled  AEGIS  software  maintenance  and  upgrade 
process  would  allow  developers  to  incorporate  increased  capabilities  into  software 
updates  sooner,  possibly  leading  to  improved  mission  effectiveness.  This  topic  could 
present  an  area  for  future  research. 

The  potential  for  OA  reengineering  exists  in  the  decomposed  off  ship  AEGIS 

software  maintenance  and  upgrade  process  model.  Many  of  the  processes  involved  in  the 
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off  ship  portion  could  be  refined,  but  due  to  data  collection  challenges,  this  research  was 
forced  to  focus  on  the  aggregate  on  ship  and  off  ship  process  model.  The  implementation 
of  OA  along  with  distance  support  policies  and  the  Maintenance  Free  Operating  Period 
(MFOP)  concept  could  drastically  alter  the  decomposed  off  ship  AEGIS  software 
maintenance  and  upgrade  process  model.  Data  capturing  methods  could  be  put  into  place 
to  provide  historical  data  inputs  along  with  subject  matter  expert  (SME)  inputs.  The 
potential  for  increased  efficiency  in  this  process  should  not  be  overlooked  and  would 
provide  an  area  for  further  research.  Another  area  for  future  research  could  employ 
business  process  reengineering  (BPR)  to  the  decomposed  off  ship  AEGIS  software 
maintenance  and  upgrade  process  in  order  to  reduce  redundancies  that  occur  between  the 
software  contractor  and  government  agencies. 

The  implementation  of  an  OA  enabled  AEGIS  software  maintenance  and  upgrade 
process  presents  an  interesting  challenge  in  the  area  of  training  shipboard  operators. 
When  a  new  software  version  is  fielded  via  distance  support  in  real  time,  no  time  period 
currently  exists  for  familiarization  and  training  with  the  new  software  version.  This 
could  potentially  have  adverse  affects  on  mission  effectiveness.  An  area  for  future 
research  exists  in  the  potential  for  distance  support  training  or  other  training  methods 
with  OA  enabled  systems. 

The  potential  for  a  move  to  Service  Oriented  Architecture  (SOA)  also  exists  for 
the  AEGIS  software  maintenance  and  upgrade  process.  This  move  would  require  a 
radical  reengineering  of  the  current  process  and  sub  process  and  could  have  the  potential 
to  greatly  improve  efficiency  by  loosely  coupling  AEGIS  modules.  A  Knowledge  Value 
Added  (KVA)  analysis  could  be  conducted  in  a  future  study  to  examine  the  potential 
benefits  of  incorporating  SOA  into  the  AEGIS  software  maintenance  and  upgrade 
process. 

Lastly,  a  Real  Options  (RO)  analysis  will  be  conducted  in  future  research.  This 
analysis  will  serve  to  project  potential  benefits  and  risks  to  different  options  that  could  be 
implemented  when  reengineering  the  AEGIS  software  maintenance  and  upgrade  process. 


72 


Real  Options  analysis  will  provide  investment  options  through  careful  analysis  of  the 
KVA  and  will  take  into  account  risk  identification,  quantification,  valuation,  mitigation 
and  diversification. 
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APPENDIX  37 


KVA+RO  Framework 

KVA+RO  measures  operating  performance,  cost-effectiveness,  return  on  investments 
(ROI),  risk,  real  options  (capturing  strategic  flexibility),  and  portfolio  optimization. 
KVA+RO  results  empower  decision-makers  and  support  IT  acquisition  business  cases 
by  providing  performance-based  data  and  scenario  analysis.  Analyses  like  the  ROI  on 
individual  projects,  programs,  processes  and  sub-processes  within  a  portfolio  of  IT 
acquisitions  can  be  derived  through  the  KVA  methodology. 

Figure  1 

Valuation  Framework 

Data  Collection  +  KVA  Methodology  +  Real  Options  Analysis  =  Historical,  Performance-Based 

Data  and  Analyses 

Providing  ROI  and  total  strategic  options  values  along  with  the  risk  measurements  for  each  option. 


37  Komoroski,  Christine  L.  2006;  4-6. 
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Knowledge  Value  Added  (KVA) 

KVA  measures  the  value  provided  by  human  capital  assets  and  IT  assets  for  an 
organization,  process  or  function  at  any  level  of  analysis.  It  monetizes  the  outputs  of  all 
assets,  including  intangible  knowledge  assets  using  a  market  comparables  valuation 
technique.  Capturing  the  value  embedded  in  an  organization’s  core  processes, 
employees  and  IT  enables  the  actual  cost  and  revenue  of  a  product  or  service  to  be 
calculated. 


Figure  2 

Measuring  Output 


Total  value  is  captured  in  the  key  metric  measurement  of  ROI.  This  ratio  has 
comparable  revenue  -  investment  cost  in  the  numerator  and  investment  cost  in  the 
denominator. 

Table  1 
KVA  Metrics 


Metric 

Description 

Type 

Calculation 

Return  on  Investment 
(ROI) 

Same  as  ROI  at  the  sub¬ 
corporate,  process  level 

Traditional  investment 

finance  ratio 

(Revenue-Investment  Cost) 

Investment  cost 

Real  Options  (RO) 

Potential  strategic  investment  options  can  then  be  evaluated  with  real  options 
analysis  using  historical  data  provided  by  KVA.  The  analysis  applied  is  a  robust  and 
analytical  process  incorporating  the  risk  identification  (applying  various  sensitivity 
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techniques),  risk  quantification  (applying  Monte  Carlo  simulation),  risk  valuation  (real 
options  analysis),  risk  mitigation  (real  options  framing),  and  risk  diversification  (analytical 
portfolio  optimization). 
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